Strommer Computing Service – 1900 S Charles Blvd., # 7C – Greenville, NC 27858 Multimedia Computing; Computer hardware and software consulting; WWW consulting; statistical data analysis July 2012 Arrow Educational Products requested data analysis of Data obtained from three studies of the use of its educational software in the North Carolina Public Schools We looked at the results of three studies Our detailed findings are contained in the attached pages A summary of our findings is shown below Study 1: Reading with Phonics with Fun and Spanish speaking English as a Second Language Students This study was done during the 2011 – 2012 school year It took a group of 92 Spanish speaking English as a Second Language Students of various abilities and ages The students were divided into two groups The control group (37 students) was taught using traditional ESL techniques The treatment group (55 students) used Reading With Phonics is Fun in addition to more traditional techniques These students were tested both prior to the experimental period and at the end of the experimental period using traditional ESL testing materials These tests grouped the students into categories based on their English proficiency level The study measured how many proficiency levels each student gained The results showed that 70% of the control group gained proficiency level or less while nearly 64% of the treatment group gained at least proficiency levels This is a very striking difference Study 2: Phonics Voyage with third graders – School This study used two classes of third graders at the same elementary school One class of twenty-six students used the Phonics Voyage reading program while another class of twenty-two students used more traditional methods The score of each student on the reading portion of the North Carolina End of Grade test was then compared with that same student’s reading score on the second grade EOG the previous year This study was carried out during the 2006 - 2007 school year The results showed an average gain of 9.3 points by the Phonics Voyage group and only 3.5 points by the control group We performed a Two-Sample t-Test on the data We performed the test twice with slightly different hypotheses The first time, the test was performed against the hypothesis that there was not truly any difference in the true average gain by the two groups The test showed that there is a 0.01% chance that the true means of the two groups are the same and that the observed difference was due to sampling uncertainty We then tested the results against the hypothesis that there should have really only been a point difference in the means as opposed to the observed 5.5 point difference The probability of this happening due to the uncertainties of sampling is 2.7% So, at the 5% level of certainty, we feel confident that the Phonics Voyage group showed a true gain of at least points more than the control group Since this is about the same as the observed gain by the control group, we feel confident in saying that the Phonics Voyage group’s gain (on the average) was at least twice that of the control group, even after the uncertainties of sampling are taken into account Of interest is that one sample t-test techniques show that there is about a 23% chance that the control group did not actually have any improvement in their scores and that the observed improvement is entirely due to sampling uncertainties There is only a 1% chance that the Phonics Voyage group actually had no increase in score and that the observed improvement was due to sampling variability Study 3: Phonics Voyage with third graders – School This study also used two classes of third graders at the same elementary school One class of seventeen students used the Phonics Voyage reading program while another class of eighteen students used more traditional methods The score of each student on the reading portion of the North Carolina End of Grade test was then compared with that same student’s reading score on the second grade EOG the previous year This study was carried out during the 2006 – 2007 school year The results showed an average gain of 16.2 points by the Phonics Voyage group and only 6.2 points by the control group We performed a Two-Sample t-Test on the data We performed the test three times with slightly different hypotheses The first time, the test was performed against the hypothesis that there was not truly any difference in the true average gain by the two groups The test showed that there is a 0.008% chance that the true means of the two groups are the same and that the observed difference was due to sampling uncertainty We then tested the results against the hypothesis that there should have really only been a point difference in the means as opposed to the observed 10 point difference The probability of this happening due to the uncertainties of sampling is 2% We did a third t-test against the hypothesis that the true difference in the gains was 6% The test showed that there was a 4.8% probability of this So, at the 5% level of certainty, we feel confident that the Phonics Voyage group showed a true gain of at least points more than the control group Since this is about the same as the observed gain by the control group, we again feel confident in saying that the Phonics Voyage group’s gain (on the average) was at least twice that of the control group, even after the uncertainties of sampling are taken into account Of interest is that one sample t-test techniques again show that there is about a 22% chance that the control group did not actually have any improvement in their scores and that the observed improvement is entirely due to sampling uncertainties This time there is only a 0.8% chance that the Phonics Voyage group actually had no increase in score and that the observed improvement was due to sampling variability So, again we feel confident in saying that the Phonics Voyage group must have had real test score gains, while the observed gains for the control group could have been paper gains caused by sampling variability The bottom line: From a statistical point of view, all three studies showed that the group using the computer software had test score gains which were at least twice as much as any gains posted by the non-computer groups About Us: Strommer Computing Service is a sole proprietorship which exists for convenience in billing the consulting activities of Dr Thomas O Strommer Dr Strommer is currently a member of the faculty of the mathematics department of East Carolina University in Greenville, NC He holds a Ph.D in mathematics from the University of Washington in Seattle, WA He has been teaching and conducting research at the university level for more than twenty-five years He has been an active consultant for more than fifteen years His interests include multimedia computing, computer database analysis and design, computer hardware and software consulting, statistical data analysis, course and test design and delivery, discrete mathematics, and geometry His consulting activities have included the computer, health care, publishing, and transportation industries