Ed 603 - Lesson 14 - Threats to Experimental Validity

Lesson 14 will consist of the following topics

· Text Assignment for Lesson 14

· Threats to the Validity of Research

· Threats to Internal Validity

· Threats to External Validity

· Lesson 14 Assignment

· Lesson 14 Quiz

Text Assignment for Lesson 14

For lesson 14, read pages 359-368 in Educational Research: Competencies for Analysis and Application, Seventh Edition by L. R. Gay and Peter Airasian (2003, Merrill/Prentice Hall, ISBN 0-13-099463-4)

or read pages 237-246 in Educational Research: Competencies for Analysis and Application, Eighth Edition by Lorrie Gay, Geoffrey Mills, and Peter Airasian (2006, Prentice-Hall, ISBN 0-13-118534-9)

or read pages 307-310 in Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, 3^rd Edition by John W. Creswell (2008, Merrill/Prentice Hall, ISBN-13: 978-0-13-613550-0)

Threats to the Validity of Research

Researchers who have evaluated experimental research have described a number of factors that can limit the validity of a research study. The validity of a study is related to the design of the experiment and the research can modify the effects of these factors on the research by selecting a research design that guards against the threats to validity.

The threats to experimental validity can be classified under two major headings 1) threats to internal validity, and 2) threats to external vality.

A study is internally valid when the the results or effects on the dependent variable are attributable to the independent variable and not to other factors. How well these other factors are controlled is related to the internal validity of the study.

External validity, sometimes called ecological validity, is concerned with the generalizability of the study. The degree to which the results of the study can be generalized to groups beyond the study sample is an indication of the external validity of the study.

Threats to Internal Validity

The text for the course discusses threats to internal validity in detail (Gay & Airasian, 2000, pages 372-376) and summarizes them in Table 11.1 (page 376).

History - Unexpected events occur between the pre- and posttest, affecting the dependent variable.
Maturation - Changes occur in the participants, from growing older, wiser, more experienced, etc. during the study.
Testing - Taking a pretest alters the result of the posttest.
Instrumentation - The measuring instrument is changed between pre- and posttesting, or a single measuring instrument is unreliable.
Statistical Regression - Extremely high or extremely low scorers tend to regress to the mean on retesting.
Differential Selection of Participants - Participants in the experimental and control groups have different characteristics that affect the dependent variable differently.
Mortality - Different participants drop out of the study in different numbers, altering the composition of the treatment groups.
Selection-Maturation Interaction - The participants selected into treatment groups have different maturation rates. Selection interactions also occur with history and instrumentation.

Threats to External Validity

The text for the course discusses threats to external validity in detail (Gay & Airasian, 2000, pages 377-383) and summarizes them in Table 11.2 (page 383).

Pretest-Treatment Interaction - The pretest sensitizes participants to aspects of the treatment and thus influences posttest scores.
Selection-Treatment Interaction - The nonrandom or volunteer selection of participants limits the generalizability of the study.
Multiple Treatment Interference - When participants receive more than one treatment, the effect of prior treatment can affect or interact with later treatments, limiting generalizability.
Specificity of Variables - Poorly operationalized variables make it difficult to identify the setting and procedures to which the variables can be generalized.
Treatment Diffusion - Treatment groups communicate and adopt pieces of each other's treatment, altering the initial status of the treatments comparison.
Experimenter Effects - Conscious or unconsious actions of the researcher affects participants' performance and responses.
Reactive Effects - The fact of being in a study affects participants from their normal behavior. The Hawthorne and John Henry effects are reactive responses to being in a study.