The Value of Technology Assessments

As technology becomes increasingly incorporated into the framework of education, the need for some sort of measurement of what our students know and are able to do relating to technology becomes more apparent. Even more apparent is the need to measure the knowledge and skills of educators and leaders who must be prepared to take the students on this educational journey.

In 2001, the U.S. Department of Education passed the No Child Left Behind act, which requires states to demonstrate that "every student is technologically literate by the time the student finishes the eighth grade, regardless of the student's race, ethnicity, gender, family income, geographic location, or disability." This requirement has put a new emphasis on technology education in our classrooms. The International Society for Technology in Education (ISTE) has published the National Educational Technology Standards (NETS) for students, teachers, and administrators, which sets the standard for what each of these groups needs to know and be able to do to be technologically literate.

Having a set of standards gives us something we can measure ourselves against. The problem comes in developing assessment tools which accurately measure the actual results against the expected learning outcomes. Consider the problem of assessing student learning outcomes for technology, given that every classroom is differently equipped, with different teachers, under different leadership, using technology in different ways. In "A Framework for Considering Technology's Effectiveness" (1998) Kathleen Fulton states, "...to ask if technology works is almost the equivalent of saying "Do textbooks work?" Yes, some textbooks “work,” in some conditions, with some teachers, with some students, but these same textbooks may not “work” in another educational context. Clearly the question of technology effectiveness requires us to be clear in what results we seek, how we measure success, and how we define effectiveness."

Standardized tests are insufficient methods to address many of the skill sets required by ISTE. "The benefits of information technology" by J. Kosakowski, (1998) states that "Educators may find impediments to evaluating the impact of technology. Such impediments include lack of measures to assess higher-order thinking skills, difficulty in separating technology from the entire instructional process, and the outdating of technologies used by the school." We are making progress in measuring some of desired learning outcomes set by ISTE, but many are difficult to measure. According to "Critical Issue: Using Technology to Improve Student Achievement" by Margaret Honey, Katherine McMillan Culp, and Robert Spielvogel (2005), "Researchers are also making progress on the more complicated task of investigating the impact of technology use on higher order thinking skills as measured through means other than standardized tests. They are examining students' ability to understand complex phenomena, analyze and synthesize multiple sources of information, and build representations of their own knowledge. At the same time, some researchers are calling for newer standardized assessments that emphasize the ability to access, interpret, and synthesize information."

It is going to be necessary to find ways to assess these and other skills to determine areas where we are meeting the standards, and areas that we still fall short. As we strive to move forward in this generation of technological literacy, we may need to look toward the very tools we are teaching for answers to the problem of how to assess the effectiveness of our teaching.