National Academies Press: OpenBook

The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries (2013)

Chapter: IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.

« Previous: IDR Team Summary 1: Develop innovative curricula that will help students develop expertise in dealing with the information overload they will encounter during and after their schooling.
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

IDR Team Summary 2

Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.

CHALLENGE SUMMARY

Over the last one-hundred years, a series of new technologies have each promised to revolutionize teaching. Slides, film and videos were all plausible technologies for replacing the classic form of instruction, in which a teacher lectures a class. Despite the many new technologies that might have changed teaching, conventional classrooms are still dominant. The World Wide Web—with the ease of posting videos and designing interactive online tests—is an attractive mélange of technologies that offer up considerable potential for influencing and improving education. Will things be different this time? How will we know the efficacy of the new Web-based teaching and learning technologies?

One reason why the Web technologies may be different is that many, many more people are authors. Pelli and Bigelow estimate that the number of authors is growing tenfold each year. The new transformation extends widespread literacy to widespread authorship. This will have an enormous impact on the range of available educational material or at least material that could be useful in educational settings.

A second reason is that Web usage, unlike slides or videos, is integrated into the typical day of many people. By integrating educational materials into daily communication and social networking tools, the new technologies have advantages that were not part of the earlier innovations.

Education is based on testing, credentialing, and building affinity groups. The Web is likely to be able to perform each of these functions, but the way in which it does so will differ from traditional classroom teaching.

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

How can we measure whether the new ways are effective? How can we use these measures to guide and improve the efficacy of these technologies over time? The variety of uses of Web technologies for teaching, for learning, and for research across the disciplines is impressive, but how can we assure that there is sharing and promulgation of those uses for re-use and re-mixing among the disciplines and learning environments?

Key Questions

• What can we learn from the history of introducing new technology into Education? Is there any difference with the new technologies that suggest we will have different outcomes?

• Are we prepared to measure the efficacy of the new technologies and use this ability to nudge them into useful directions?

• New software applications and new possibilities arising from them are being promoted to help authors produce educational material. Will this increase authorship and change the range of materials? What is the reasoning at academic and textbook publishers and in the software industry?

• In the US and many other Western countries, education and research are completely intertwined. What are the implications for research Universities if the education model is transformed?

• What are the implications for middle school and high school if we rely on instructional materials that are distributed, used, and take advantage of Web technologies? What is the new role for teachers and teacher education? Does it change dramatically? What is the uptake of such technologies in our public schools? Has the use of modern information technology improved teaching and learning in the K-12 grades?

Reading

Bhattacharjee Y, Clery D, Dede C, Greenfield PM, Hines DJ, Jasny BR, Mervis J, Miller G, Normile D, et. al. Special science online collection: Education and technology. Science (Special Issue) 2 January 2009;323(5910). Gates Foundation. Next generation learning: The intelligent use of technology to develop innovative learning models and personalized educational pathways. 2010.

Khan S. Let’s use video to reinvent education. You Tube (Video) March 2011.

Nochese F. Khan Academy: My final remarks. Action-reaction: Reflections on the dynamics of teaching 10 May 2011.

Pelli DG and Bigelow C. Nearly universal literacy is a defining characteristic of today’s modern civilization; nearly universal authorship will shape tomorrow’s. Seed Magazine 20 October 2009.

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

Robinson K. Schools kill creativity. TED Presentation 2006 June.

Stanford University. Stanford University EPGY.

Thompson C. How Khan Academy is changing the rules of education. Wired Magazine 15 July 2011.

Thrun S. University 2.0. Livestream (Video) 2012.

Watters A. The wrath against Khan: Why some educators are questioning Khan Academy. Hack Education 19 July 2011.

Widom J. From 100 Students to 100,000. ACM Sigmod 24 February 2012.

IDR TEAM SUMMARY

Zahra Hirji, NAKFI Science Writing Scholar Massachusetts Institute for Technology

IDR Team 2 was charged with the herculean task of developing methods to efficiently design and measure the efficacy of various Internet teaching technologies. Today’s teachers and self-learners have more tools at their disposal than at any previous point in history, in large part due to the proliferation of the World Wide Web. Out of all the Internet tools now available—video-audio lecture surrogates, adaptive tutorials, games, online quizzes/assessments, wikis, blogs, chat systems—how does one possibly decide which tool or tools to adopt?

To make informed decisions, teachers and self-learners need access to technology evaluation data. This information would ideally be housed in a single location and consist of standardized assessments that allow for technology comparisons. Unfortunately these data and infrastructure are largely nonexistent. As a result, users are forced to select tools based on the hyperbole of marketing rather than on scientifically rigorous assessment.

IDR Team 2 outlined the actions needed to give teachers and users pursing online education, whether formal or informal, whether for themselves or to teach others, access to the assessment information they deserve and need. The envisioned end goal also includes equipping technology users with the resources to conduct their own qualified assessments.

The Internet Is Different

The current mania surrounding Web-based teaching technologies is nothing new. Back in the 1870s, Boston schools thought the school slate— personalized chalkboards—would revolutionize education. Similar claims were then made about the film projector, radio, television, and laptop.

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

Educators are always looking to improve student performance, and the untapped potential of new technologies is deliciously appealing.

Schools have adopted these various technologies to different degrees, but there is little proof that such tools actually improve student performance over prolonged periods of time. As a result, the way education is generally conducted has not been dramatically altered in modern history. From K-12 to higher education, teachers and professors still commonly lecture in front of the classroom, hand out paper exams, and employ textbooks for homework.

So, is the Internet actually any different? The team definitively decided yes. Unlike previous technologies, Web-based ones can build upon the strengths of older technologies. Televisions brought pre-recorded visual material into education, but unless you had a TV and school videos at home, that technology was confined to the classroom. But using the Web, videos can be made accessible at any time—in the classroom or outside the classroom—to anyone using a computer, smartphone, or tablet.

These modern technologies also break the barrier between the classroom and world with the capacity to “inreach,” or bring untapped information or people (parents, alumni, other teaching professionals) into the classroom in a virtual context. In addition to stimulating better learners, these tools may benefit teachers by automating normally exhaustive tasks, such as grading homework or tests, which then frees up the teacher’s time for more constructive personal interactions with students. These tools can also help teachers track the progress or tailor workloads of individual students more readily. Finally, these technologies benefit those outside formal education too, where it is now easier to refresh old skills or pick up new ones on your own.

Is the Internet a Threat?

Internet tools are flexible, interactive, and adaptive technologies—and can be accessed from the privacy of your own living room. Moreover training teachers in new technologies is time and cost intensive. If these are so great, accessible, seemingly independent, do we even need teachers or traditional classrooms? Now this may seem like a bold question, but many state universities in the process of slashing funding are turning to massive open online class [MOOC] alternatives, such as Coursera and Open Learning Initiative, to replace, or at least supplement, traditional lectures. While this has not yet corresponded with the widespread layoffs of associate

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

or adjunct professors, the general mood among some college campuses has been described as tense, to put it mildly.

Despite the appeal of MOOCs, there is little evidence that such technologies are just as effective, or better than traditional lecture classes. For the few subjects where evidence does exist and is relatively positive, mainly for some math and science courses, it is not yet possible to extrapolate whether this teaching format will work for other subjects, such as the social sciences and humanities.

Many of the team members are professors, and a few even had experience with MOOCs. For the MOOC users, the online portion of the class served primarily as a supplement to in-class time rather than a complete alternative. In other words, these technologies were job-helpers, not job-eaters. Regardless of the future role of MOOCs in the greater education systems, the ability to assess online technologies comprehensively is paramount. The last thing society want is the complete eradication of in-class learning for sub-par online technologies.

What Criteria, Information, and Tools Are Necessary for Assessment?

When approaching the task of building useful web teaching technology assessments, it is necessary to review what criteria, information, and tools are necessary for assessment—and how much of that material is currently known or available. The team members realized there is much we do not know or have at this point in time. In response, the team identified the three major challenges: (1) knowledge of diverse stakeholder needs, (2) a lack of accessible student performance data (at an individual or even aggregate level) for different technologies, (3) and no existing cohesive framework and infrastructure for assessing and evaluating learning outcomes for such technologies.

Assessment criteria

Ideally, Internet education technology assessments should meet the criteria of diverse stakeholders—K-12 teachers, professors, self-learners, and even employers. Some general student performance assessment points include fact learning, life skills, student engagement, student retention and completion, student diversity and access, course cost, and attitudes toward lifelong learning. Is it possible for any one assessment to actually evaluate all the criteria listed, or will separate specific assessments be more useful, practical?

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

Data collection

In order to assess diverse performance measures, there needs to be available data on student performance, ideally measured continually or at least routinely throughout the learning process. Many existing tools do not collect or share these data with the public. Of the ones that do both, educators can still only access very aggregated information. This is one of the major charges against the popular online software Khan Academy. Despite its popularity and plethora of testimonials, the company does not make its data accessible to the public for external evaluation.

On the flip side, the mentality that all data are good data is not necessarily true (unless you are a researcher). If educators could identify metrics for which they specifically want data, such as student quiz and assessment scores and time spent per online homework assignment, then existing Internet technologies could be reprogrammed to collect tailored information that meet stakeholder specific needs.

Assessment framework & infrastructure

There is no existing framework and infrastructure for assessing and evaluating learning outcomes for these modern technologies. In fact, there are few relevant frameworks for non-Internet based education technologies. One notable example, however, is the online resource What Works. This website is a database of researched and reviewed educational programs, products, practices, and policies for the K-12 school system. The website’s existing information is high quality, but there is relatively little site content. Moreover, this resource does not currently account for Web-based teaching technologies.

Recommendations

The research team came up with two sets of recommendations to pursue the advancement of World Wide Web teaching technology assessment: the development of assessment infrastructure and the proposal of research topics to take advantage of this new infrastructure.

Building appropriate infrastructure

The team decided a future Internet teaching technology assessment infrastructure should be modeled after What Works. This resource would be

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

a more scaled-up comprehensive version of What Works, including a public database of teaching technology assessments, relevant for K-12, higher education, and life-learning needs. All available assessments would be a certain high quality and have to go through some level of review. Moreover, assessments would ideally be standardized to allow for technology comparisons. In addition, there should be automated tools available for good experiment and assessment design so that website users could contribute their own experiences and feedback.

Future research goals

Once this new database is constructed, various research studies using the assessment data could provide valuable information. Below are five potential research questions:

• What is the right technology for a given goal or learner?

• How do learner characteristics, such as media proficiency, mediate the effectiveness of Internet teaching technologies?

• What is the right mixture of technology and social interactions, whether face-to-face or virtual to benefit individuals and society as a whole?

• How can we anticipate the effect of new technologies?

• What are novel designs for technologies that promote learning?

Internet technologies do not just offer an alternative to classroom learning, but a potential to substantially change education. Only with additional research on technology assessment and learner preferences might the idea of Internet-driven education become a significant part of the educational system.

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×

This page intentionally left blank.

Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 21
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 22
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 23
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 24
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 25
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 26
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 27
Suggested Citation:"IDR Team Summary 2: Develop methods to efficiently design and measure the efficacy of Internet teaching technologies.." National Research Council. 2013. The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries. Washington, DC: The National Academies Press. doi: 10.17226/18268.
×
Page 28
Next: IDR Team Summary 3: Define the trajectory, value, and risk of Extreme Lifelogging when nearly everything about a person is in Cyberspace. »
The Informed Brain in a Digital World: Interdisciplinary Research Team Summaries Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Digital media provide humans with more access to information than ever before—a computer, tablet, or smartphone can all be used to access data online and users frequently have more than one device. However, as humans continue to venture into the digital frontier, it remains to be known whether access to seemingly unlimited information is actually helping us learn and solve complex problems, or ultimately creating more difficulty and confusion for individuals and societies by offering content overload that is not always meaningful.

Throughout history, technology has changed the way humans interact with the world. Improvements in tools, language, industrial machines, and now digital information technology have shaped our minds and societies. There has always been access to more information than humans can handle, but the difference now lies in the ubiquity of the Internet and digital technology, and the incredible speed with which anyone with a computer can access and participate in seemingly infinite information exchange. Humans now live in a world where mobile digital technology is everywhere, from the classroom and the doctor's office to public transportation and even the dinner table. This paradigm shift in technology comes with tremendous benefits and risks. Interdisciplinary Research (IDR) Teams at the 2012 National Academies Keck Futures Initiative Conference on The Informed Brain in the Digital World explored common rewards and dangers to Humans among various fields that are being greatly impacted by the Internet and the rapid evolution of digital technology.

Keynote speaker Clifford Nass of Stanford University opened the dialogue by offering insight into what we already know about how the "information overload" of the digital world may be affecting our brains. Nass presented the idea of the "media budget," which states that when a new media emerges, it takes time away from other media in a daily time budget. When additional media appear and there is no time left in a person's daily media budget, people begin to "double book" media time. Personal computers, tablets, and smartphones make it easy to use several media simultaneously, and according to Nass, this double-booking of media can result in chronic multitasking, which effects how people store and manage memory. Although current fast-paced work and learning environments often encourage multitasking, research shows that such multitasking is inefficient, decreases productivity, and may hinder cognitive function. National Academies Keck Future Initiative: The Informed Brain in a Digital World summarizes the happenings of this conference.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!