Research papers with wrong statistical methods
What Makes a Statistical Analysis Wrong?
The P value cannot say this: It may be the first statistical term to rate a definition in the online Urban Dictionary, where the usage examples are telling: The effect had disappeared, and with it, Motyl's dreams of youthful fame 1. They're actually handed down to us by ourselves, through the methodology we adopt. Statistical methods in anesthesia. My harangue today is on testing or statistical significance, about which Bill Rozeboom wrote 33 years ago, "The statistical folkways of a more primitive past continue to dominate the local scene. Fourth, there were three cases where the error of comparing the means on the categorical variables was made. Sensitive to controversies over reproducibility, Motyl and his adviser, Brian Nosek, decided to replicate the study.
Right—most of the time, there is not just one Right. But there are many that are clearly Wrong.
Luckily, what makes an analysis right for your data is more easily defined than what makes a person right for you. It pretty much comes down to two things: Assumptions are very important.
A test needs to reflect the scale of the variables, the study design, and issues in the data. A repeated measures study design requires a repeated measures analysis.
Korean J Med Educ. Instead of doing four separate small studies and reporting the results in one paper, for instance, researchers would first do two small exploratory studies and gather potentially interesting findings without worrying too much about false alarms. Many are not ashamed and some seem proud to research papers with wrong statistical methods that they 'don't know anything about statistics'. Critics also bemoan the way that P values can encourage muddled thinking.
A binary dependent variable requires a categorical analysis method. But within those general categories, there are often many analyses that meet assumptions.
- Methodological and statistical techniques:
- Research findings from underpowered, early-phase clinical trials would be true about one in four times, or even less frequently if bias is present.
- A growing body of literature points to persistent statistical mistakes, flaws and deficiencies in most medical journals" Strasak et al.
A logistic regression or a chi-square test can both handle a binary dependent variable if there is only a single categorical predictor. But a logistic regression can also incorporate covariates, directly test interactions, and calculate predicted probabilities.
A chi-square test can do none of these. So you get different information from different tests. They answer different research questions.
And Research papers with wrong statistical methods mean writing them in minute detail. Issues of mediation, interaction, subsetting, control variables, et cetera, should all be blatantly obvious in the research questions. Thinking about how to write results before solidifying the research questions ensures the analysis is able to answer the questions.
Whether the answer research papers with wrong statistical methods what you expected or not is a different issue. Want to ask an expert all your burning stats questions?