As Deborah Stirling (2002) points out, “Many teachers attempting to integrate new technologies like software programs into their teaching practice rely on evaluations to help them make instructional decisions.” It is therefore crucial that the appropriate stakeholders supply a thorough overview of given software, its potential and limitations. The software evaluators will include experts from the field of study. The experts will be comprised of the lead evaluator, the technical expert of the school or district who will have the role of identifying and reporting on the technical aspects of the software. Secondly, there should be a subject area expert who ensures that curriculum concerns are being reached with the new software. At a later point, student users will be given the opportunity to use the software and to provide user-feedback via surveys and participation in a focus group. It is important that those students clearly represent the age and ability level of those who will actually utilize the software as a learning resource.
In Kenneth Komoski’s (1995)Seven Steps to responsible Software selection, “Factors to consider in specifying requirements for software include: compatibility with available hardware; cost ; user friendliness; level of interaction desired; adequacy of documentation; access to technical support via toll-free number; and of course, direct correlation with the instructional objectives and curriculum requirements identified in the needs analysis.”(1995) The most important consideration being the ability of the software to be an effective tool in the meeting of curricular objectives.
For the purpose of this section, the general outline of the model is such that it may be adapted to any curriculum, providing that the evaluators have express knowledge of, and access to, the curriculum outcomes.
Technical Expert Rationale
The technical expert is an important member in a software evaluation’s team. Technical experts are those individuals who are well informed and qualified to make comments and judgments related to the technical and logistical function of software in the school setting. Schools sometimes have a technical expert on staff. That person will probably be the resource person, or teacher, who administers the school’s computer network, or the person who oversees many of the technical decision making at the school level. Many school districts have technical experts who handle the technical issues on behalf of schools and teachers.
The technical expert’s software evaluation should be completed and reported prior to the commencement of any other evaluations of the software. It is important to know the technical limitations and potential of the product before too much time is invested upon its evaluation. Many of the criteria the technical expert uses are non-critical, they just state a fact. However, if some of the criteria elicit possible weaknesses in the software then those weaknesses need to be addresses. For example, if the minimum system requirements are just not feasible at the school level then the software, no matter how highly it rates by the other evaluators, will never be implemented at that school level. Or, if the overall cost is just too high then further evaluation will probably be done in vain.
Technical experts will be expected to consult the other expert evaluators and maybe even the student evaluators to gain insight into how the software is to be utilized at the school level. The technical evaluation and summary will possibly assist the other evaluators with their task of evaluation and provide the potential for a more thorough and critical review of the software.
Subject Area Expert Rationale
The subject area expert would provide the most important evaluation of the software in terms of the educational value it contributes the teaching-learning environment. The expert could be a teacher at the school intending to use the software, a program specialist at a school district office or Department of Education, or an educator with a background in the subject area being addressed. The subject area expert would not include a person associated with a software publishing or distribution company.
Learner Representatives: Student Users
This model provides opportunity for a representative sample (between three and seven individuals) to be drawn from the user population (ideally a class of about 20). Gaining user feedback through surveys and a focus group will greatly increase validity of an evaluator(s)’s decisions. Indeed, without user feedback, there is a risk that the evaluation will be quite the opposite. Reiser and Dick (1990) clearly state that “Without testing students after they use the software, judgments regarding the instructional effectiveness of software are primarily speculative.” Results of their experiments indicated that negative evaluations were given subjectively to programs which yielded positive results in field tests and those which received positive evaluations often yielded negative field results. This indicates a need to include learner representative feedback (for this purposes of an evaluation, these represent field tests with actual users). It does not, however, suggest that expert evaluations are invalid.