Part IV: Benefits in Education

Fortunately for our discussion, the academics have been pretty interested in the issue for a long time. Distance education has been with us since television, computers, having been developed largely in academic centers, naturally found their way into the classroom. I can remember seeing them in my elementary school around 1976. Of course one can't compare the machines of yore to today's desktop, and early efforts, weather beneficial or not, have been swept aside by new generations of software and hardware. However, perhaps in an inverted extension of Pinkett's critical mass thesis, things seems to be stabilizing to a certain degree. I can attribute this is the fact the transmission architecture (from a data perspective) and the hardware it relies upon has not only reached a certain state of ubiquity, but it's standardized open-architecture has allowed for universal compatibility in many measures. In a way, this ubiquity has reached a critical mass which makes it impractical to consider changing it regardless of many benefits that might be attainable by doing so. There is also a simple fact: it's still just a media and there's only so much you can do with it. For all intents and purposes, the internet, as experienced from a desktop PC, started out as an extension of the printed page. Then it started to wiggle, jiggle and make noise. What it's ended up being is an extension of the printed page which wiggles, jiggles and makes noise, that can stand in for your TV or entertainment center, which is interactively plugged into the largest single body of information which is accessible anywhere on earth and which is by the way, accessible from any place on earth, at least in theory. It has become a communication device that has added vast dimension to instant communications and may in fact supplant traditional service mediums such as analog telephony. But it is still, quite simply, a communications device.

I would briefly frame this discussion by referencing the recent paper Fool's Gold: A Critical Look at Computers in Childhood, Alliance For Childhood, Sept, 2001, Anshel, Armstrong, Bowers, Godnig, Landis, Kane, Monke, Moody, Noble, Noorani, Orr, Talbott, Sclove, Winner, which maintains that there are negligible educational benefits to the very young resulting from technology, say to those 1-5 years of age. I do not wish to contest this at all, and instead focus on older youth, those who are preparing for the imminent assumption of adult responsibilities or those who have already taken them on. And although Fool's Gold is highly qualitative and anecdotal it reinforces what one would hope would be instinctively grasped: what benefit would placing a toddler in front of a monitor have over human contact? I leave the parsing of this sub-issue to interested parties, assuming that these same include no parents, and will restrict further discussion to individuals 10 years of age and older.

The Pew Partnership Research

Moving into realms governed by common sense as well as sound observation, Pew Charitable Trusts does a nice job of surveying school kids and their parents in The Internet and Education: Findings of the Pew Internet & American Life Project (September 2001, Amanda Lenhart, Maya Simon, Mike Graziano). This research shows that teens use the Internet as an essential study aid outside the classroom and that the Internet plays an increasing role in the classroom. Their survey consisted of 754 internet-using youths ages 12-17 and one of their parents or guardians between November 2, 2000 and December 15, 2000. This sample size does not allow us to draw conclusive determinations about general US usage patterns for this group and the final data were not weighted. For results based on this survey as indicated in it's report, the margin of error is plus or minus 4 percentage points. These results are nonetheless used to approximate numbers of Americans who access the Internet. These approximations are derived from the Census Bureau's estimates of the number of adults living in US telephone households.

94% of youth ages 12-17 who have Internet access say they use the Internet for school research and 78% say they believe the Internet helps them with schoolwork.

71% of online teens say that they used the Internet as the major source for their most recent major school project or report.

41% of online teens say they use email and instant messaging to contact teachers or classmates about schoolwork.

34% of online teens have downloaded an online study aid.

18% of online teens say they know of someone who has used the Internet to cheat on a paper or test.

58% of online teens report using Web sites that had been set up specifically for their school or for a particular class.

17% of online teens have created a Web page for a school project.

Key findings from a survey of 754 of the parents of those youths:

87% of parents of online teens believe that the Internet helps students with their schoolwork and 93% believe the Internet helps students learn new things.

55% say the Internet has been a good thing for their children; only 6% say it has been a bad thing; 38% say the Internet has no effect one way or the other on their children.

55% say that it is essential for today's children to learn how to use the Internet in order to be successful and another 40% believe it is important.

28% of these parents have used email to communicate with their children's teachers.

Other education-related findings in the survey:

5% of adult Internet users have taken a class online for college credit. That amounts to more than 5 million people.

53% of adult Internet users have gone online to do research for school or job training. On any given day 8% of adult Internet users are online doing research for school or job training.

52% of adult Internet users have done job-related research online. On a typical day, 16% of Internet users are online doing job-related research. About half of Internet users (more than 50 million people) have access to the Internet at work. Source:

The Pew report gives us valuable insight and hints at the information rich online environment that induces students to utilize it and further hints at the ease with which students in this age groups are achieving comfort with this new medium. It also fleshes out one of out earlier points, the importance and function of home use in educational applications. Home use clearly plays a vital role in the educational experience of the young people in this study as reported by participants and their parents. The report is still conspicuously qualitative in it's findings and far short of controlled evidence in support of program validity.

The CAST Study

A study by CAST (Center for Applied Special Technology), "The Role of Online Communications in Schools: A National Study" demonstrates in more concise terms and with greater control that students with online access perform better and measures some effects on learning in the classroom. The study compared the work of 500 fourth and sixth graders in 28 elementary and middle school classes in 7 urban school districts (Chicago, Dayton, Detroit, Memphis, Miami, Oakland, and Washington DC), half with online access and half without. Both groups (those with and those without internet access) received the same curricula focusing on the subject of civil rights for the same simultaneous duration. With this controlled process, CAST provides reasonable evidence that access to the internet and technologies, if properly integrated, can make a positive impact. In each test an experimental and control fourth grade class from one school, and an experimental and control sixth grade class from the other school were tested. Only the experimental classes were allowed to take part in online resources including Scholastic Network access, activities, and communication although duplicate materials existed for offline classes where appropriate. Additionally, student sand teachers in all groups were encouraged to make use of multimedia and other technology assets.

In both groups, emphasis was placed on integrated learning, i.e. helping students to see connections among people, their actions, and the real world. The classroom approach was to break students out into smaller groups which would study according to group preference, then to participate in a set of activities which included everyone in the study. The three major learning activities were communications (discussing civil rights issues with peers, teachers, family members, neighbors, people in the community), research (using a variety of media to explore and synthesize information from multiple sources) and creation of a final project, a common part of both fourth and sixth grades curriculum.

Student learning was determined upon final projects evaluation. These evaluations were based on 9 learning measures. Students were required to demonstrate knowledge, apply skills, and illustrate their thinking processes. Additionally, they were required to maintain journals and worksheets to provided a forum for participant-evaluation. The learning measures used to assess the final projects were built into the worksheets so that students became familiar with them.

The student's work was evaluated and graded by a third party reviewers, an experienced school teacher trained in assessment methods. The evaluator had no prior knowledge of either the study or any of the participating schools and was instructed to score the projects fairly and without bias. The work was also reviewed by a CAST researcher. Following this, over a quarter of the projects were randomly selected (half from experimental classes, half from control classes) and scored by a second external evaluator. Correlations of the two external evaluators' scores were used to substantiate the consistency of the main evaluator's project ratings. The 9 rating points were:

The results show significantly higher scores on measurements of information management, communication, and presentation of ideas for experimental groups with online access than for control groups with no online access. It offers evidence that using Scholastic Network and the Internet can help students become independent, critical thinkers, able to find information, organize and evaluate it, and then effectively express their new knowledge and ideas in compelling ways. This is a fairly innovative method when considering the many challenges to control in such a broad-based study. However, the review and grading process, relying upon a single 3rd party reviewer and a single researcher, albeit confirmed by a third 3rd party reviewer, leaves us lacking the kind of universality achieved by the NTIA reports.

The CAST study introduces certain aspects of programmatic development which are oft-cited but have been relatively unsubstantiated to date in scientific terms, namely that we cannot expect predictable or beneficial results from teaching tools which are not adequately integrated into the programmatic environment or which do not at least seek to achieve isolated or rudimentary goals. Most educators have intuitively understood technology to be an educational asset, not an educational replacement and with good reason: technology does not create content, be it data or epistemology or ideas, it merely allows us to present or access it in various nontraditional ways.

But as noted previously, many capable educators have been set to the task of quality programmatic development, to demonstrating beneficial uses of computers in education and to a thoughtful and informed integration of computers into curricular contexts and environments.

The Wenglinski Study

These points are most authoritatively made in Does It Compute? The Relationship Between Educational Technology and Student Achievement In Mathematics, Harold Wenglinsky Policy Information Center 1998, provides findings that not only demonstrate benefits on a more universal scale with a far greater sample size, but he also draws the distinctions of programmatic integration quite clearly into perspective. In fact, Wenglinski is able to demonstrate negative effects of technology-enhanced learning if it is improperly applied in the context of the same test.

It is with Wenglinski that we see the best demonstrations not only of variable degrees of access and programmatic integration but of educator training. His report presents findings from a national study of the relationship between different uses of educational technology and various educational outcomes. Data were drawn from the 1996 National Assessment of Educational Progress (NAEP) in mathematics, consisting of national samples of 6,227 fourth graders and 7,146 eighth-graders. I summarize the key findings below:

Among fourth-graders, 33.2 percent reported using computers in school for mathematics once week or more. Black students had slightly higher use rates than other groups at 41.9% reporting frequent use as compared to 32.3 percent of Hispanic students and 31.7 percent of White students. Students in the Southeast used computers in schools more frequently than Midwest students. Public school students used computers more frequently than students attending private school. There were no differences based upon gender, economic status, or community status.

Eighth- graders showed comparable computer usage as fourth graders at 28.3% using them once a week or more. There were no regional differences. Males had higher levels of computer use than females. Poor students had higher levels of computer use than non-poor students. Students in urban schools had higher levels of computer use than students attending rural schools. Private school eighth graders showed higher levels of computer use than public school students.

As such, these findings show traditionally disadvantaged groups no longer lagging in access to school computers as of 1996 in the terms that we have focused on so far. The limited scope of this study makes it difficult to compare to broader national figures such as we had seen in the NTIA reports, but the findings are nonetheless fascinating. From here, however, Wenglinksi's analysis is more telling.

58% of fourth-graders have access to a computer at home but access is variable between groups. Two-thirds of White and Asian students had access to computers at home as opposed less than half of Black and Hispanic students. Less than half of the students attending schools in the Southeast had access to a computer at home compared to two-thirds of Northeast students.

Eighth-graders showed the same basic pattern. Overall, 64.3 percent reported a computer at home. Black and Hispanic students were much less likely to have access than White and Asian students. Students from the Northeast had higher levels of access than students from the Southeast. Private school students had higher levels of access than public school students. Non-poor students had higher levels of access than poor students.

More public school teachers than private school teachers reported professional training with technologies, as did more teachers of non-poor students than teachers of poor students, and more suburban teachers than urban rural teachers. The Southeast led with 85.6% of teachers followed the West and the Midwest and the Northeast with 77.0%. There were no ethnic or gender differences. For eighth grade, the percentage of students with teachers reporting professional development was 76%. There were no significant differences based upon ethnicity, school governance, or economic status.

54.5 percent of fourth-graders have teachers reporting learning games as the primary use, 35.9 percent report drill and practice; 7.5 percent report limitations and applications; and 2.1 percent report introducing new topics. Wenglinsky thus finds that higher-order skills such as applying concepts or developing simulations to illustrate them are rarely used. 34.3 percent teachers for eighth-graders report drill and practice as the primary use. Playing learning games is the second most popular use, followed closely by applications (29.2 percent and 27.2 percent respectively). Teachers report introducing new topics as the primary use .2 percent of the time. Southeastern students are less likely to play learning games than they are to use computers for drill and practice. Wenglinski found no significant differences based upon ethnicity, gender, or economic status. Eighth graders showed differences in use for all categories except gender. Blacks were less likely to use computers for applications and more likely to use them for drill and practice. Use varied with region as well. The Southeast was the most likely to drill and the least likely to do applications; the Midwest was the most likely to do applications and the least likely to drill. Also, suburban and non-poor students were more likely to use computers for applications; public school students were more likely to use computers for drill and practice. In eighth grade, minority, poor, and urban students were more likely to learn lower-order skills than their White, non-poor, and suburban counterparts; disadvantaged students were also less likely to find themselves learning higher order skills. (Note that the above paragraph so closely resembles the original text as to border upon plagiarism. The original is so efficient, however, as to warrant little change in paraphrase.)

As such, Wenglinksi found that the greatest inequities did not lie in how often computers were used, but in how and where they were used. Technology could matter, but that this depended upon how it was used. Specifically for eighth-graders:

The teacher's professional development in technology and the use of computers to teach higher-order thinking skills were both positively related to academic achievement in mathematics and the social environment of the school.

The frequency of home computer use was positively related to academic achievement and the social environment of the school.

The use of computers to teach lower-order thinking skills was negatively related to academic achievement and the social environment of the school.

The frequency of school computer use was unrelated to the social environment of the school and negatively related to academic achievement.

For fourth-graders, the study found that using computers for learning games was positively related to academic achievement

Taken more generally, Wenglinski's arguments support the findings of the CAST study, which sought factors external to access in order to explain benefits-or lack of same-of technologies in classrooms. In each case, we find that computers must be integrated into the programmatic environment, that drill-and-practice applications are of limited value, that the instructor's expertise with technology and that more qualified evaluations of technology presence are all critically relevant factors. One might easily surmise that the application of technology to the curricular environment will not be a one-size-fits-all, blanket application, but that all traditional effects such as age and development path stand fully in effect.

The No Significant Difference Phenomenon

This site provides selected entries from the book "The No Significant Difference Phenomenon" which links to studies that find no significant difference between performance of in-person as opposed to video distance learning classes.

The significant difference companion website has links to studies finding significant differences in same.

See also:

Evaluating Educational Technologies, Evaluation of Educational Technology Initiatives. California Instructional Technology SRI Center for Technology in Learning, Trochim Evaluating Web Sites, EnGauge, NCREL The Secretary's Conference on Educational Technology Evaluating the Effectiveness of Technology, ERIC Clearinghouse on Assessment and Evaluation, Brown University Scholarly Technology Group.

Forward to next page
Back to Contents
Back to top