Mixing
“Liberal†p-values?
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
9
down vote
favorite
My question is rather semantic. When a method routinely produces high p-values it is called conservative. Would you call the opposite, i.e. a method with a high type-II-error rate liberal?
hypothesis-testing statistical-significance p-value terminology type-i-and-ii-errors
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
asked Aug 6 at 12:48
JonJup
524
add a comment |Â
up vote
9
down vote
favorite
My question is rather semantic. When a method routinely produces high p-values it is called conservative. Would you call the opposite, i.e. a method with a high type-II-error rate liberal?
hypothesis-testing statistical-significance p-value terminology type-i-and-ii-errors
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
asked Aug 6 at 12:48
JonJup
524
4
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
2
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56
add a comment |Â
up vote
9
down vote
favorite
up vote
9
down vote
favorite
My question is rather semantic. When a method routinely produces high p-values it is called conservative. Would you call the opposite, i.e. a method with a high type-II-error rate liberal?
hypothesis-testing statistical-significance p-value terminology type-i-and-ii-errors
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
asked Aug 6 at 12:48
JonJup
524
My question is rather semantic. When a method routinely produces high p-values it is called conservative. Would you call the opposite, i.e. a method with a high type-II-error rate liberal?
hypothesis-testing statistical-significance p-value terminology type-i-and-ii-errors
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
asked Aug 6 at 12:48
JonJup
524
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
edited Aug 6 at 15:07
Harvey Motulsky
10.2k44181
10.2k44181
asked Aug 6 at 12:48
JonJup
524
asked Aug 6 at 12:48
JonJup
524
asked Aug 6 at 12:48
JonJup
524
524
4
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
2
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56
add a comment |Â
4
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
2
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56
4
4
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
2
2
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
7
down vote
accepted
According to this homepage it is common to use this terminology.
Conservative in statistics has the same general meaning as in other areas: avoiding excess by erring on the side of caution. In statistics, “conservative†specifically refers to being cautious when it comes to hypothesis tests, test results, or confidence intervals. Reporting conservatively means that you’re less likely to be giving out the wrong information.
which can be specified in the following sense:
A conservative test always keep the probability of rejecting the null hypothesis well below the significance level. Let’s say you’re running a hypothesis test where you set the alpha level at 5%. That means that the test will (falsely) give you a significant result 1 out of 20 times. This is called the Type I error rate. A conservative test would always control the Type I error rate at a level much smaller than 5%, which means your chance of getting it wrong will be well below 5% (perhaps 2%).*
However I recommend you to use other terminologies, e.g. the definition of power. If a hypothesis test is "liberal" in your terminology it has more power. If a hypothesis test is "conservative" in your terminology it has less power. In my experience the term "a liberal hypothesis" is barely used in practice and might sound uncommon to your audience even if your audience consists of statisticians.
In the following paragraph I explain why "conservativ" and "liberal" are not always the exact difference in politics. Therefore I disrecommend using liberal as opposite of conservative in statistics. Feel free to ignore this part if it does not help you
Note that also in political science liberal is not necessarily the opposite of conservative. In the US left-wing politicians like Bernie Sanders are called liberals, but in many parts of Europe, e.g. Germany, the Netherlands and Denmark it is different. In German Politics Liberalism is mainly understand as the maximum of political freedom, especially in economics. The German Liberal Party (FDP) is in many issues rather right-wing than socialist even though they endorse issues like LGBT Rights and the Legalisation of Cannabis. Some Germans might think of what is called Libertarian in the US when you mention "liberal politics". In Denmark and the Netherlands it is even more complicated. You have two big parties which consider themself as liberal- In the Netherlands "VVD" and "D66"; In Denmark the "Vestre" and the "Radicale Vestre". While "VVD" and "Vestre" are rather "right-wing" the "D66" and the "Radicale Vestre" are rather left wing.
For this reason you should not use the terminology: "conservative statistical test" and "liberal statistical test" when speaking to a global, international audience.
PS: I hope I kept my political stance out of this topic and explained it neutrally.
answered Aug 6 at 13:20
Ferdi
3,18331948
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
 |Â
show 1 more comment
up vote
0
down vote
The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.
Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.
The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).
answered Aug 8 at 14:48
DavidR
1,347814
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
7
down vote
accepted
According to this homepage it is common to use this terminology.
Conservative in statistics has the same general meaning as in other areas: avoiding excess by erring on the side of caution. In statistics, “conservative†specifically refers to being cautious when it comes to hypothesis tests, test results, or confidence intervals. Reporting conservatively means that you’re less likely to be giving out the wrong information.
which can be specified in the following sense:
A conservative test always keep the probability of rejecting the null hypothesis well below the significance level. Let’s say you’re running a hypothesis test where you set the alpha level at 5%. That means that the test will (falsely) give you a significant result 1 out of 20 times. This is called the Type I error rate. A conservative test would always control the Type I error rate at a level much smaller than 5%, which means your chance of getting it wrong will be well below 5% (perhaps 2%).*
However I recommend you to use other terminologies, e.g. the definition of power. If a hypothesis test is "liberal" in your terminology it has more power. If a hypothesis test is "conservative" in your terminology it has less power. In my experience the term "a liberal hypothesis" is barely used in practice and might sound uncommon to your audience even if your audience consists of statisticians.
In the following paragraph I explain why "conservativ" and "liberal" are not always the exact difference in politics. Therefore I disrecommend using liberal as opposite of conservative in statistics. Feel free to ignore this part if it does not help you
Note that also in political science liberal is not necessarily the opposite of conservative. In the US left-wing politicians like Bernie Sanders are called liberals, but in many parts of Europe, e.g. Germany, the Netherlands and Denmark it is different. In German Politics Liberalism is mainly understand as the maximum of political freedom, especially in economics. The German Liberal Party (FDP) is in many issues rather right-wing than socialist even though they endorse issues like LGBT Rights and the Legalisation of Cannabis. Some Germans might think of what is called Libertarian in the US when you mention "liberal politics". In Denmark and the Netherlands it is even more complicated. You have two big parties which consider themself as liberal- In the Netherlands "VVD" and "D66"; In Denmark the "Vestre" and the "Radicale Vestre". While "VVD" and "Vestre" are rather "right-wing" the "D66" and the "Radicale Vestre" are rather left wing.
For this reason you should not use the terminology: "conservative statistical test" and "liberal statistical test" when speaking to a global, international audience.
PS: I hope I kept my political stance out of this topic and explained it neutrally.
answered Aug 6 at 13:20
Ferdi
3,18331948
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
 |Â
show 1 more comment
up vote
7
down vote
accepted
According to this homepage it is common to use this terminology.
Conservative in statistics has the same general meaning as in other areas: avoiding excess by erring on the side of caution. In statistics, “conservative†specifically refers to being cautious when it comes to hypothesis tests, test results, or confidence intervals. Reporting conservatively means that you’re less likely to be giving out the wrong information.
which can be specified in the following sense:
A conservative test always keep the probability of rejecting the null hypothesis well below the significance level. Let’s say you’re running a hypothesis test where you set the alpha level at 5%. That means that the test will (falsely) give you a significant result 1 out of 20 times. This is called the Type I error rate. A conservative test would always control the Type I error rate at a level much smaller than 5%, which means your chance of getting it wrong will be well below 5% (perhaps 2%).*
However I recommend you to use other terminologies, e.g. the definition of power. If a hypothesis test is "liberal" in your terminology it has more power. If a hypothesis test is "conservative" in your terminology it has less power. In my experience the term "a liberal hypothesis" is barely used in practice and might sound uncommon to your audience even if your audience consists of statisticians.
In the following paragraph I explain why "conservativ" and "liberal" are not always the exact difference in politics. Therefore I disrecommend using liberal as opposite of conservative in statistics. Feel free to ignore this part if it does not help you
Note that also in political science liberal is not necessarily the opposite of conservative. In the US left-wing politicians like Bernie Sanders are called liberals, but in many parts of Europe, e.g. Germany, the Netherlands and Denmark it is different. In German Politics Liberalism is mainly understand as the maximum of political freedom, especially in economics. The German Liberal Party (FDP) is in many issues rather right-wing than socialist even though they endorse issues like LGBT Rights and the Legalisation of Cannabis. Some Germans might think of what is called Libertarian in the US when you mention "liberal politics". In Denmark and the Netherlands it is even more complicated. You have two big parties which consider themself as liberal- In the Netherlands "VVD" and "D66"; In Denmark the "Vestre" and the "Radicale Vestre". While "VVD" and "Vestre" are rather "right-wing" the "D66" and the "Radicale Vestre" are rather left wing.
For this reason you should not use the terminology: "conservative statistical test" and "liberal statistical test" when speaking to a global, international audience.
PS: I hope I kept my political stance out of this topic and explained it neutrally.
answered Aug 6 at 13:20
Ferdi
3,18331948
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
 |Â
show 1 more comment
up vote
7
down vote
accepted
up vote
7
down vote
accepted
According to this homepage it is common to use this terminology.
Conservative in statistics has the same general meaning as in other areas: avoiding excess by erring on the side of caution. In statistics, “conservative†specifically refers to being cautious when it comes to hypothesis tests, test results, or confidence intervals. Reporting conservatively means that you’re less likely to be giving out the wrong information.
which can be specified in the following sense:
A conservative test always keep the probability of rejecting the null hypothesis well below the significance level. Let’s say you’re running a hypothesis test where you set the alpha level at 5%. That means that the test will (falsely) give you a significant result 1 out of 20 times. This is called the Type I error rate. A conservative test would always control the Type I error rate at a level much smaller than 5%, which means your chance of getting it wrong will be well below 5% (perhaps 2%).*
However I recommend you to use other terminologies, e.g. the definition of power. If a hypothesis test is "liberal" in your terminology it has more power. If a hypothesis test is "conservative" in your terminology it has less power. In my experience the term "a liberal hypothesis" is barely used in practice and might sound uncommon to your audience even if your audience consists of statisticians.
In the following paragraph I explain why "conservativ" and "liberal" are not always the exact difference in politics. Therefore I disrecommend using liberal as opposite of conservative in statistics. Feel free to ignore this part if it does not help you
Note that also in political science liberal is not necessarily the opposite of conservative. In the US left-wing politicians like Bernie Sanders are called liberals, but in many parts of Europe, e.g. Germany, the Netherlands and Denmark it is different. In German Politics Liberalism is mainly understand as the maximum of political freedom, especially in economics. The German Liberal Party (FDP) is in many issues rather right-wing than socialist even though they endorse issues like LGBT Rights and the Legalisation of Cannabis. Some Germans might think of what is called Libertarian in the US when you mention "liberal politics". In Denmark and the Netherlands it is even more complicated. You have two big parties which consider themself as liberal- In the Netherlands "VVD" and "D66"; In Denmark the "Vestre" and the "Radicale Vestre". While "VVD" and "Vestre" are rather "right-wing" the "D66" and the "Radicale Vestre" are rather left wing.
For this reason you should not use the terminology: "conservative statistical test" and "liberal statistical test" when speaking to a global, international audience.
PS: I hope I kept my political stance out of this topic and explained it neutrally.
answered Aug 6 at 13:20
Ferdi
3,18331948
According to this homepage it is common to use this terminology.
Conservative in statistics has the same general meaning as in other areas: avoiding excess by erring on the side of caution. In statistics, “conservative†specifically refers to being cautious when it comes to hypothesis tests, test results, or confidence intervals. Reporting conservatively means that you’re less likely to be giving out the wrong information.
which can be specified in the following sense:
A conservative test always keep the probability of rejecting the null hypothesis well below the significance level. Let’s say you’re running a hypothesis test where you set the alpha level at 5%. That means that the test will (falsely) give you a significant result 1 out of 20 times. This is called the Type I error rate. A conservative test would always control the Type I error rate at a level much smaller than 5%, which means your chance of getting it wrong will be well below 5% (perhaps 2%).*
However I recommend you to use other terminologies, e.g. the definition of power. If a hypothesis test is "liberal" in your terminology it has more power. If a hypothesis test is "conservative" in your terminology it has less power. In my experience the term "a liberal hypothesis" is barely used in practice and might sound uncommon to your audience even if your audience consists of statisticians.
In the following paragraph I explain why "conservativ" and "liberal" are not always the exact difference in politics. Therefore I disrecommend using liberal as opposite of conservative in statistics. Feel free to ignore this part if it does not help you
Note that also in political science liberal is not necessarily the opposite of conservative. In the US left-wing politicians like Bernie Sanders are called liberals, but in many parts of Europe, e.g. Germany, the Netherlands and Denmark it is different. In German Politics Liberalism is mainly understand as the maximum of political freedom, especially in economics. The German Liberal Party (FDP) is in many issues rather right-wing than socialist even though they endorse issues like LGBT Rights and the Legalisation of Cannabis. Some Germans might think of what is called Libertarian in the US when you mention "liberal politics". In Denmark and the Netherlands it is even more complicated. You have two big parties which consider themself as liberal- In the Netherlands "VVD" and "D66"; In Denmark the "Vestre" and the "Radicale Vestre". While "VVD" and "Vestre" are rather "right-wing" the "D66" and the "Radicale Vestre" are rather left wing.
For this reason you should not use the terminology: "conservative statistical test" and "liberal statistical test" when speaking to a global, international audience.
PS: I hope I kept my political stance out of this topic and explained it neutrally.
answered Aug 6 at 13:20
Ferdi
3,18331948
edited Aug 7 at 6:49
answered Aug 6 at 13:20
Ferdi
3,18331948
answered Aug 6 at 13:20
Ferdi
3,18331948
answered Aug 6 at 13:20
Ferdi
3,18331948
3,18331948
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
 |Â
show 1 more comment
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
2
2
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
Semi-off-topic aside: "In the US left-wing politicians like Bernie Sanders are called liberals" – this is changing as well in the US. In at least certain wings, "establishment" Democrats are called "liberal" whereas more "radical"/"progressive" figures like Bernie, the Democratic Socialists of America, etc. are termed "left" but not "liberal."
– Dougal
Aug 6 at 13:46
2
2
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
You could probably do without the section that describes the political usage of the words "conservative" and "liberal". It was somewhat distracting and not exactly on topic.
– indigochild
Aug 6 at 17:17
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
"A conservative test always keep the probability of rejecting the null hypothesis well below the significance level." That doesn't make sense. The probability of rejecting H0, given that it's true, is the significance level.
– Acccumulation
Aug 6 at 18:29
1
1
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@indigochild I separated my post in the "core answer" and an addendum. I hope this makes it more readible.
– Ferdi
Aug 6 at 18:38
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
@Acccumulation. The goal is what you say, to design a test so the probability of rejecting H0, given that it's true, is the significance level. But this isn't always possible. A "conservative" test is designed so the probability of rejecting the null hypothesis is less than (or equal to) the significance level.
– Harvey Motulsky
Aug 7 at 18:20
 |Â
show 1 more comment
up vote
0
down vote
The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.
Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.
The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).
answered Aug 8 at 14:48
DavidR
1,347814
add a comment |Â
up vote
0
down vote
The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.
Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.
The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).
answered Aug 8 at 14:48
DavidR
1,347814
add a comment |Â
up vote
0
down vote
up vote
0
down vote
The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.
Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.
The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).
answered Aug 8 at 14:48
DavidR
1,347814
The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.
Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.
The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).
answered Aug 8 at 14:48
DavidR
1,347814
answered Aug 8 at 14:48
DavidR
1,347814
answered Aug 8 at 14:48
DavidR
1,347814
answered Aug 8 at 14:48
DavidR
1,347814
1,347814
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f360936%2fliberal-p-values%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
4
Definitely not, if only because the term is so vague as to be meaningless. (I wouldn't use "conservative," either, without first explaining what I meant.)
– whuber♦
Aug 6 at 12:51
2
A method can be conservative without routinely producing high p-values. If the null is never true and you never take small samples the p-values may routinely be low but the method may still be conservative. Its opposite is sometimes called 'anticonservative'
– Glen_b♦
Aug 6 at 15:44
The cumulative probability of getting a p-value of p0 is, by definition, p0 (for the continuous case). Every method has the same chance of producing high p-values as every other method.
– Acccumulation
Aug 6 at 18:33
The term p-value is already doing enough damage when such concepts as 'conservative p-values' are around. P-values are used much too 'liberal' leading to all kinds of practices, conclusions and decisions which they should not. The p-value is a rule-of-thumb-measure of distance. When we wish to challenge a hypothesis then it is good to be 'conservative' (it makes us look further and better). I do not see the use of 'liberal' (too low) estimates of p-values unless we actually (a priori) believe it is good to reject the hypothesis and aim for power (which you dont necessarily get with liberal).
– Martijn Weterings
Aug 8 at 12:53
How about neurotic for a method that keeps producing high p-values
– Aksakal
Aug 8 at 14:56