Pre-Experimental Designs or Weak Experimental Designs
Louise H. Kidder, L. "Research Methods in Social Relations", Holt-Saunders International Editions (1980), identifies the following pre-experimental designs:
a) The One-Shot Case Study
For example, you want to determine whether praising primary school children makes them do better in arithmetic. You measure mathematics achievement with a test. To test this idea, you choose a class of year 2 pupils and increase praising of children and you find that their mathematics score did increase. You conclude that praising children, increases their mathematics score.
X O
(praise) (attentiveness)
This design is weak for the following reasons:
1) Selection: It is possible that the students you selected as
subjects were already good in mathematics.
2) History: The school had organised a motivation course on
mathematics for year 2 students. So, it is possible it
might influence their performance.
Experimental Research Designs
Symbols Used in Experimental Research Design
Research Designbe thought of as the structure of research, i.e. it is the 'glue' that holds all of the elements in a research project together. In experimental research, a few selected symbols are used to show the design of a study.
O = Observation or Measurement (eg. mathematics score, score
on an attitude scale, weight of subjects, etc.)
O1, O2, O3 = more than one observation or measurement.
R = Random assignment: subjects are randomly assigned to the
various groups.
X = Treatment which may be a teaching method, procedure in
counseling, reading strategy, frequency of questioning, etc.
b) The One-Group Pretest-Posttest Design
To ensure that there was no pre-existing characteristic among the pre-school children, a pretest may be administered. If the children became more attentive after praising compared to the pretest, then you can attribute it to the practice of praising.
O1 X O2
(pretest) (praise) (posttest)
This design is weak for the following reasons:
1) Maturation: If time between the pretest and posttest is long,
it is possible that the subjects may have matured becasue
of developmental changes.
2) Testing: Sometimes the period between the pretest and the
posttest is too short and there is the possibility that
subjects can remember the questions and answers.
c) The Static-Group Comparison Design or Nonequivalent Posttest-Only
The dashed lines separating the experimental group and the control group indicates that the children were not randomly assigned to the two groups. Unlike the earlier designs, an attempt was made to include a comparison group (i.e. control group) that did not receive 'praise'. However, subjects were not randomly assigned and there is no indication that the two groups are equivalent (called non-equivalent groups).
This design is weak for the following reason:
1) Selection: Since there was no random assignment, it cannot be established that the two groups are equivalent. So, any differences in the posttest may not be attributable to giving praise but other factors such as ability, IQ, interest, ect.
The Experimental Method: Hypothesis Testing & Weak Research Designs
The purpose of an experiment is to test a hypothesis or hypotheses. An experiment starts with The EXPERIMENTAL HYPOTHESIS which is a prediction that a certain treatment (eg. inductive approach) will cause a particular effect (eg. enhanced creative thinking abilities).
b) Null Hypothesis
Opposite to the experimental hypothesis is the NULL HYPOTHESIS which states that there is no evidence that the treatment (eg. inductive approach) has a effect (eg. enhanced creative thinking abilities). In other words, any difference that existed between the experimental group and the control group is DUE TO CHANCE. The Null hypothesis states that there is no difference between the two groups and in statistical terms it is as follows:
Ho : M a = M b (Equation A)
Ho : M a - M b = 0 (Equation B)
Ho = notation for the Null Hypothesis. You could have H1, H2, H3 and so on; which simply means Hypothesis 1, Hypothesis 2, Hypothesis 3 and so forth.
M a = Mean for Group 1
M b = Mean for Group 2
So for Equation A, the mean score for Group 1 is more less EQUAL to the mean score of Group 2. and there is no statistical significance difference, then and the Null hypothesis is ACCEPTED.
Equation B is similar. The mean of Group 1 minus the mean of Group 2 is equal to 0, indicating no significant differences and the Null Hypothesis is ACCEPTED.
If the means of the two groups are different and there is a statistical difference, then you REJECT the null hypothesis.
what is an experiment
the purpose of an experiment
the role of random assignment
how to ensure the groups compared are equivalent
the importance of internal validity and how it can be threatened
the importance of external validity and how it can be jeopardised
If you DO NOT KNOW the following,
Test of Significance
In order to enable you to reject the null hypothesis, it is necessary to analyse the data statistically. Why is this necessary? For example, in your experiment you obtained the following:
Mean Std. Deviation
Experimental Group 30.4 3.7
Control Group 28.3 4.1
To the naive person, he or she would conclude that the experimental group performed better than the control group because the mean score is higher by 2.1 and that the treatment is effective. This is misleading because it is likely that the differences in the mean between the experimental and control group could have occured by chance. In order for you to accept or reject the null hypothesis, it is necessary that you analyse the data statistically because you want to be sure that the treatment administered produced a real effect.
How do you determine that the difference between the two groups is caused by the treatment and not some other extraneous variable? You could repeat the experiment and see if you get the same results which will provide evidence of the reliability of the obtained findings. However, this is not an economical approach and for this reason statistical tests are preferred.
The test of significance enables one to determine whether the amount of difference between the two groups is due to chance or due to the treatment. Does a large difference between the mean score of the experimental and control group indicate that the difference is real? Even large differences could occur by chance, although the probability of this happening would be very low. The most common practice is to state a significance level that must be reached; which is a statement of the probability that an observed difference is chance difference. The most common significance levels are .05 and .01; regardless whether you are using the t-test, F-test or the chi-square.
If you decide from the onset of the experiment that the .05 significance level is to be used, it means that you will accept as a real difference only one that is so large that it could have occured by chance only 5 times in 100 (i.e. 95% not due to chance). If the .01 significance level is selected, then the difference can be expected to occur only 1 time in 100 by chance (i.e. 99% not due to chance).
In order to enable you to reject the null hypothesis, it is necessary to analyse the data statistically. Why is this necessary? For example, in your experiment you obtained the following:
