Describe the greatest threats to internal validity and how you would mitigate them. Also describe the greatest threats to external validity and how you would address them.

Threats to Internal Validity of Research Design

1. History: - this is the events that is observed between first and second measurements.

2. Instrumentation: - this is difference that occurs to the results due to inconsistency of the instruments

3. Maturation: - this means the longer the research, the higher the chances of sample to have outgrown or improved in their performance.

4. Testing

5. Selection Biases

6. Experimental Mortality

How to mitigate the Internal Validity threats

1. History: - Quickly identify external factors that may influence the results.

2. Instrumentation: - Ensure there is consistency on the instruments.

3. Maturation: - Reduce time if possible between pretest and post test period.

Threats to External Validity of Research Design

1. Multiple Treatment Interference: - conducting multiple treatment of the same sample are difficult to control for the effect of prior treatment.

2. Interaction Effect of Testing: - a previous test tends to influence samples' response to the variables used in experiment.

3. Interaction Effects of Selection Biases

4. Reactive Effects of Experimental Arrangements

How to mitigate the threats to External Validity of Research Design

1. Interaction effect of testing: - Avoid interaction with the sample before the test.

2. Interaction effects of selection biases: - Make use of randomization when selecting samples.

Explanation:

Internal Validity of Research Design simply means that, whether or not, the experimental conditions/treatments make difference, and whether the available evidence/results support such fact or claim being considered.

While

External Validity of research design is a means of using the outcomes / results of a particular study/research to determine cases outside that study. That is, the application of the research result to other situations or issues that need be considered.

Internal validity refers specifically to whether an experimental treatment/condition makes a difference or not, and whether there is sufficient evidence to support the claim.

External validity refers to the generalisation of treatment/condition outcomes.

Explanation:Greatest threat to Internal Validity:

History--the specific events which occur between the first and second measurement.

Maturation--the processes within subjects which act as a function of the passage of time. i. e. if the project lasts a few years, most participants may improve their performance regardless of treatment.

Testing--the effects of taking a test on the outcomes of taking a second test.

Instrumentation--the changes in the instrument, observers, or scorers which may produce changes in outcomes.

Statistical regression--It is also known as regression to the mean. This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment.

Selection of subjects--the biases which may result in selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, when the sample size is small, randomization may lead to Simpson Paradox, which has been discussed in an earlier lesson.

Experimental mortality--the loss of subjects. For example, in a Web-based instruction project entitled Eruditio, it started with 161 subjects and only 95 of them completed the entire module. Those who stayed in the project all the way to end may be more motivated to learn and thus achieved higher performance.

Selection-maturation interaction--the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect.

John Henry effect--John Henry was a worker who outperformed a machine under an experimental setting because he was aware that his performance was compared with that of a machine.

Greatest threat to External Validity:

Interaction of testing and X--because the interaction between taking a pretest and the treatment itself may effect the results of the experimental group, it is desirable to use a design which does not use a pretest.

Interaction of selection and X--although selection is controlled for by randomly assigning subjects into experimental and control groups, there remains a possibility that the effects demonstrated hold true only for that population from which the experimental and control groups were selected. An example is a researcher trying to select schools to observe, however has been turned down by 9, and accepted by the 10th. The characteristics of the 10thschool may be vastly different than the other 9, and therefore not representative of an average school. Therefore in any report, the researcher should describe the population studied as well as any populations which rejected the invitation.

How to address internal threat:

History--this is controlled in that the general history events which may have contributed to the O1 and O2 effects would also produce the O3 and O4 effects. This is true only if the experiment is run in a specific manner--meaning that you may not test the treatment and control groups at different times and in vastly different settings as these differences may effect the results. Rather, you must test simultaneously the control and experimental groups. Intrasession history must also be taken into consideration. For example if the groups truly are run simultaneously, then there must be different experimenters involved, and the differences between the experimenters may contribute to effects.

A solution to history in this case is the randomization of experimental occasions--balanced in terms of experimenter, time of day, week and etc.

Maturation and testing--these are controlled in that they are manifested equally in both treatment and control groups.

Instrumentation--this is controlled where conditions control for intrasession history, especially where fixed tests are used. However when observers or interviewers are being used, there exists a potential for problems. If there are insufficient observers to be randomly assigned to experimental conditions, the care must be taken to keep the observers ignorant of the purpose of the experiment.

Regression--this is controlled by the mean differences regardless of the extremety of scores or characteristics, if the treatment and control groups are randomly assigned from the same extreme pool. If this occurs, both groups will regress similarly, regardless of treatment.

Selection--this is controlled by randomization.