Statistical Significance And Validity Of Research Essay

Statistical Significance And Validity Of Research Essay

Length: 1838 words (5.3 double-spaced pages)

Rating: Better Essays

Open Document

Essay Preview

Statistical significance seeking (i.e., p-hacking) is a serious problem for the validity of research, especially if it occurs frequently. Head et al. provided evidence for widespread p-hacking throughout the sciences, which would indicate that the validity of science is in doubt. Previous substantive concerns about their selection of p-values indicated they were too liberal in selecting all reported p-values, which would result in including results that would not be interesting to have been p-hacked. Despite this liberal selection of p-values Head et al. found evidence for p-hacking, which raises the question why p-hacking was detected despite it being unlikely a priori. In this paper I reanalyze the original data and indicate Head et al. their results are an artefact of rounding in the reporting of p-values.

Megan Head and colleagues (Head 2015) provide a large collection of p-values that, from their perspective, indicates widespread statistical significance seeking (i.e., p-hacking) throughout the sciences. Concerns have been raised about their selection of p-values, which was deemed too liberal and unlikely to find p-hacking to begin with (Simonsohn 2015), which raises the question why evidence for p-hacking was found by Head et al. nonetheless. The analyses that form the basis of their conclusions operate on the tenet that p-hacked papers show p-value distributions that are left skew below .05 (Simonsohn 2014). In this paper I evaluate their selection choices and analytic strategy in the original paper and suggest that Head et al. found widespread p-hacking as an artefact of rounding. Analysis files for this paper are available at

The p-value distribution of a set of heterogeneous results, as collected...

... middle of paper ...

...lyses are included in a publication arising from a single study. p-hacking has also been known about for many years; it was described, though not given that name, in 1956 (De Groot, 2014). The term p-hacking was introduced by Simonsohn, Nelson & Simmons (2014) to describe the practice of reporting only that part of a dataset that yields significant results, making the decision about which part to publish after scrutinising the data. There are various ways in which this can be done: e.g., deciding which outliers to exclude, when to stop collecting data, or whether to include covariates. Our focus here is on what we term ghost variables: dependent variables that are included in a study but then become invisible in the published paper after it is found that they do not show significant effects.acking and evidential value. PeerJ 4:e1715

Need Writing Help?

Get feedback on grammar, clarity, concision and logic instantly.

Check your paper »

Differences Between Descriptive And Inferential Statistics Essay

- Read Part G, Topics 47-61 and Part H, Topics 62-67 in our textbook. Post a reading report on this discussion board forum answering the following questions: 1. What are the differences between descriptive and inferential statistics. Descriptive stats summarize data so the data can be comprehended. The researchers prepare a frequency distribution which shows the frequencies as descriptive statistics. Percentages, and averages are also descriptive statistics. Therefore, the descriptive statistics describe sets of data collected through observation....   [tags: Statistical significance, Statistics]

Better Essays
1130 words (3.2 pages)

How Successful Are Antismoking Campaigns At Altering An Individuals Physical And Psychological Responses?

- In response to the research question, “How successful are antismoking campaigns at altering an individuals physical and psychological responses?”, the present sample size and investigation indicate that the method of advertisement does not make a significant difference in the heart rate or understanding score of participants. There is a minimal difference between central tendency values for the central and peripheral route of persuasion, with the central route having higher mean values than the peripheral route for both heart rate and understanding....   [tags: Statistics, Statistical significance]

Better Essays
1139 words (3.3 pages)

Essay on Validity And Difficulty Of Identifying Demerit

- This paper centres on the definition, validity and difficulty of identifying idiolect which is also referred to as the ‘linguistic fingerprint’ within written and spoken language. Idiolect is a fairly recent notion that is still quite vague in definition and still needs further development. The extent to which the literal definition of ‘idiolect’ can be deemed reliable within a forensic linguistic investigation is very debatable as it appears to suggest that samples of an individual’s vocabulary can be collected and measured whilst also suggesting that that person’s language use or vocabulary is fixed (Coulthard 2004)....   [tags: Linguistics, Language, Writing]

Better Essays
1105 words (3.2 pages)

A Research Study On Statistics Essay

- Statistics are tools used to collect and interpret as a way to assist in answering research questions. According to Leedy and Ormrod (2013), statistics have two functions: to describe quantitative data and then draw inferences from the data. An overwhelming amount of data can be condensed to help in determining patterns and relationships to apply to research questions. This paper will explore statistics by looking at individuals with Type 2 Diabetes in two different groups. One is an intervention group participating in a 12-week lifestyle management program consisting of regular weekly physical activity and weight loss counselling, and the other group did not receive an intervention....   [tags: Statistics, Statistical hypothesis testing]

Better Essays
916 words (2.6 pages)

A Research Study On Transformational Leadership Essay

- In the article, Onorato (2013) provides the background linking the leadership in schools to the leadership in other institutions. Based on this, the researcher develops the research problem that the research will address without using research questions. However, establishing research objectives framed in terms of research questions would be more effective in designing the research, as the research questions would distill a single piece of the broader curiosity of the researcher into a more coherent and potentially answerable thought (Gatrell, Bierly, & Jensen, 2012)....   [tags: Scientific method, Research, Research methods]

Better Essays
1290 words (3.7 pages)

Psychological Intervention Research Techniques Essay

- This assignment presents an analysis of various Intervention research techniques. Given that Randomized Control Trials are considered to be a gold standard for testing the efficiency and efficacy of interventions, the main focus of this assignment will be on the value of other types of intervention research techniques. For the sake of convenience most of the examples in this assignment will focus on interventions for schizophrenia. Schizophrenia is a disorder which is characterized by disturbances in the content of thought, perception, affect, sense of self, motivation, and behaviour....   [tags: Psychological Research Techniques]

Better Essays
3036 words (8.7 pages)

Research Methodology For Academic Journal Articles Essay

- Research methodologies are critical elements in research since it determines the type of finding at the end of the research. This paper critically outlines the different research approaches applied in academic journal articles. It includes; 1) a quantitative research approach which involves the collection and conversion of the data into the numerical forms to allow room for the statistical calculation and conclusion, 2) the qualitative research approach that involves recording, analyzing and even endeavoring to uncover a deeper meaning and the significance of the human experience and behavior (William & Laurence, 1989, p....   [tags: Scientific method, Research, Qualitative research]

Better Essays
2149 words (6.1 pages)

Evidence Based Practice : Research Design Essay

- Introduction Evidence-based practice utilizes research design analysis from arrays of studies design, it is imperative for research scholars to understand how to read, and how to comment on the quality of research design utilized. There is three major research designs, namely quantitative, qualitative and mixed method; however; mostly utilized in evidence-based practice are quantitative and qualitative. The quantitative and qualitative research designs are exclusive in the description and generate distinct outcomes....   [tags: Qualitative research, Scientific method]

Better Essays
1134 words (3.2 pages)

The Reliability and Validity of Hans Eysenck's EPI Test Essay

- The Reliability and Validity of Hans Eysenck's EPI Test The aim of this investigation was to look at the reliability and validity of Hans Eysenck's EPI Test. The EPI questionnaire comprises of items of a 'yes/no' variety. They are essentially intended as research tools (as opposed to diagnostic tools for use in clinical settings) and, as such, 'they are regarded as acceptable, reliable and valid' (Kline 1981, Shackleton and Fletcher, 1984).The Alternative Hypothesis was that there would be a positive correlation between the E and N scores for forms A and B of the EPI....   [tags: Papers]

Free Essays
2573 words (7.4 pages)

Descriptive Statistic Is A Statistic Analysis Essay

- Descriptive statistic is a statistic analysis to describe the characteristic of the respondents (Pallant, 2013). This study employs descriptive statistical analysis which gives value of mean, median and standard deviation of the respondents based on several indicators, such as sex/gender, educational level, position at work and income of the respondents. By using these indicators, the researcher describes the profile of the respondents. Hence, it can give some valuable information about the respondents....   [tags: Psychometrics, Validity, Factor analysis]

Better Essays
2111 words (6 pages)