Anybody who learns the statistics will be familiar with theconcept of type I and type II error. In hypothesis testing, Type I error, also known as a “false positive”: the error ofrejecting a null hypothesis when it is actually true. In other words, this isthe error of accepting an alternative hypothesis (the real hypothesis ofinterest) when the results can be attributed to chance. Plainly speaking, itoccurs when we are observing a difference when in truth there is none (or morespecifically - no statistically significant difference).
Type II error, also known as a "false negative": theerror of not rejecting a null hypothesis when the alternative hypothesis is thetrue state of nature. In other words, this is the error of failing to accept analternative hypothesis when you don't have adequate power. Plainly speaking, itoccurs when we are failing to observe a difference when in truth there is one. Inpractice, the statistical power (equals to 1 - type II error) is commonly used.Power is the probability of rejecting the null hypothesis when the nullhypothesis is indeed not true (i.e., the alternative hypothesis is true).
Now, there are also a concept of type III error. Fundamentally, Type III errors occurwhen researchers provide the right answer to the wrong question. Whilethe term ‘type III error’ has been used in literature and presentations, thetrue meaning of ‘type III error’ is not clearly or consistently defined. Peoplemay use the term “type III error” to refer to different things.
In one of presentations, the type III error was used todescribe those clinical trials that would have been successful but were notperformed due to resource constraint
We make type III error when conclusion is not supported by thedata
Type III error referring to an error by rejecting a nullhypothesis but inferring the incorrect alternative hypothesis.
A type III error is where you correctlyreject thenull hypothesis, but it’s rejected for the wrong reason. This compares to aType I error (incorrectly rejecting the null hypothesis) and a Type II error(not rejecting the null when you should). Type III errors are not consideredserious, as they do mean you arrive at the correct decision. They usuallyhappen because of random chance and are a rare occurrence. You can alsothink of a Type III error as giving the right answer (i.e. correctly rejectingthe null) to the wrong question. Either way, you’re stillarrivingat the correct conclusion for the wrong reason. When we say the “wrong question”, that normally meansyou’ve formulated your hypotheses incorrectly. In other words, both yournulland alternate hypothesesmay bepoorly worded or completely incorrect.
In a presentation slides titled “Type III and Type IV Errors: StatisticalDecision-Making Considerations in addition to Rejecting and Retaining the NullHypothesis”, type III error was used to refer to the wrong model, right answer andcommon influences on type III error would be:
- ·Incorrectoperationalization of variables
- ·Poor theory (e.g.,ad hoc explanations of findings)
- ·Mis-identifyingcausal architecture (Schwartz & Carpenter, 1999)
Schwartz & Carpenter (1999) TheRight Answer for the Wrong Question: Consequences of Type III Error for PublicHealth Research
In an editorial article of Arch Surg, the type III errorwas described as “the type III error occurs whenever the conclusions drawn arenot supported by the data presented”. The author presented 5 examples using thepublished articles.
Type III error is solvingthe wrong problem precisely – from Raiffa – 1968
Solving the wrong problem is defined as a Type III error byHoward Raiffa (1968, p. 264) and Ian Mitroff (1974). Type III errors aredifferent from Type I and Type II errors, which involve setting thesignificance level too high or too low in testing the null hypothesis.
in an article by Twardowski and Misra (2013) Con: Randomizedcontrolled trials (RCT) have failed in the study of dialysis methods:
“The second trial (the HEMO study) committed a Type IIIstatistical error asking the wrong question and did not bring any valuableresults, but at least it did not lead to deterioration of dialysis outcomes inthe USA”
Type III error— asking the wrong question and achieving the correctanswer:
Kimball [7] postulated a Type III error, an error thatgives the right answer to the wrong problem. A Type IV error was subsequentlypostulated as a type of error that solved the right problem too late [8].
in Flick, U. (2006). An introduction to qualitative research(3rd ed.). Thousand Oaks, CA: Sage.
Flick (2006), for example, discusses qualitative validityin terms of “whether researchers see what they think they see” (p. 371).Moreover, he and others (Kirk & Miller, 1986) argue that three types oferror may occur as regards qualitative validity: seeing a relationship, aprinciple, and so on when they are not correct (Type I error); to reject themwhen they are correct (Type II error); and asking the wrong questions (Type IIIerror).
Green and Tones (1999) “Fordebate: Towards a secure evidence base for health promotion”,
Discussion so far has essentially been concerned withassessing the outcome of interventions and has ignored the nature of theintervention itself. Type III error refers to rejection of the effectiveness ofa programme when the programme was inadequate in terms of design or delivery.This is neatly encapsulated in the acronym GIGO – garbage in, garbage out!
Tuck et al (1986) Adefence of the small clinical trial: evaluation of three gastroenterologicalstudies
The real worry clinically is the type III error, in which aclinically significantly inferior treatment is preferred to a superior one onthe basis of insufficient dataTraditional Type I and II error describe the false positiveand negative rates, Type III error describes the opportunity cost of notinvestigating valid hypotheses due to budgetary limitations
Inan article by Counsell (2002) PredictingOutcome After Acute and Subacute Stroke Developmentand Validation of New Prognostic Models
Predictor variables must beeasy to collect (to minimize missing data), clinically relevant, and reliable.Thenumber of variables in multiple regression analyses must also be carefullycontrolled.Too few variablesmeans that important predictors may be omitted, while too many variables can resultin overfitting (a type I error in which false-positive predictors areerroneously included in the model); underfitting (a type II error in whichimportant variables are omitted from the final model); and paradoxical fitting(a type III error in which a variable that, in truth, has a positiveassociation with the outcome is found to have a negative association).The riskof these problems increases as the ratio of outcome events to the number ofpredictor variables becomes smaller (the events per variable [EPV] ratio, inwhich the number of events is the lower figure for binary outcomes). The riskof error is especially high with EPVs <10 .="" span="">10>
In an article by Robin et al (1990) “Type3 and type 4 errors in the statistical evaluation of clinical trials”, the typeIII error was referred to
“Type 3 errors, then, are errors in which the risksof a given medical or public health approach is underestimated, undetected ornot specifically sought, leading to an underestimate of the risk-benefitbalance.”. The type 3 error was further classified as three categories: type 3Aerrors arise from a failure to obtain sufficient data to determine thestatistical significance of a given risk in an experimental versus a controlgroup. Type 3-B errors involve failuresto look for or detect specific risks in an experimental versus a control group.Type 3-C errors involve risks in which the harm to subjects occurs months toyears after the initial use of the modality As a result, the risk-benefit ratioof the modality is seriously underestimated.
Obviously, type III error from clinical trials has greater impacton health policy and medical practice because it involves in making the rightdecision. While the impact of type I and type II errors are the issues within aclinical trial, the impact of type III error goes beyond a clinical trial - ifa type III error is committed, we could potentially adopted a wrong practice dueto insufficient information .
