Greetings, everyone! I’m Hao, an academic cooperative researcher. In this piece, I’ll introduce a paper from our recent English literature seminar.
Paper Details:
Title: “A framework and checklists for evaluating online learning in higher education”
Journal: Assessment & Evaluation in Higher Education
Pages: Vol. 30, No. 5, October, pp. 539–553
Year of Publication: 2005
Authors: Peter Hosiea, Renato Schibecib, and Ann Backhausc
This paper delves into using checklists and contextualized assessment to evaluate online learning materials in higher education. It highlights the need for more assessment data beyond checklists, advocating for contextualized assessments. Utilizing both checklists and contextualized assessments in higher education can offer valuable insights to online learning experience designers and developers.
The paper notes the evolving technology supporting education, predicting its potential future cost-effectiveness and widespread usage. It emphasizes not only the potential for economies of scale but also the prospects of flexible delivery.
In any higher education course, evaluation should be a fundamental quality activity. However, there’s a lack of consensus among education stakeholders regarding evaluation components and methodologies, as highlighted by the authors.
The checklist in this study references categories proposed by Cheung (1994), based on OECD/CERI. Several studies have previously presented checklists for courseware reviews using OECD/CERI terminology (Sharp, 1996; Roblyer et al., 1997; Levine, 1996; Roblyer et al., 1997; Cheung, 1994).
The study emphasizes the crucial role of faculty in evaluating online classes and stresses the need for contextualized assessments aligned with the students’ needs. It presents a checklist developed from OECD/CERI, augmented with contextualized evaluations, categorized into three main sections:
1. Pedagogy: Supporting learning activities (real-world challenges, collaborative opportunities, learner-centered environment, motivating learning, meaningful assessment).
2. Resources: Content and information provided to learners (accessibility, chronology, richness, use of media, inclusiveness).
3. Delivery Strategies: How the course is delivered (reliable interface, clear goals, direction, learning plan, download efficiency, equity, accessibility, style).
The checklist aids ECU faculty and staff in assessing online unit quality, used by peer evaluators for quality assurance reviews and guidance to instructional designers in the Learning Resource Used for.
Additionally, contextualized assessment designed for the intended audience yields valuable data when combined with checklists. Student feedback indicates higher-than-average material quality, praising the learner-centered environment, engagement, richness, inclusiveness, and meaningful assessments. Areas for improvement, such as structured online discussions and maximizing “Blackboard” features, were also noted.
Future improvements must consider individual learning styles in multimedia design and emphasize learner preferences in both design and evaluation. The paper suggests challenges in designing active participant programs and flexible courseware designs.
In my reflections, I’ve been exploring a related paper for my doctoral dissertation centered around the use of checklists. What I find lacking in this paper is a deeper exploration of each checklist item. It also overlooks detailing how student comments improved the checklist evaluations. There’s a need for a more explicit correlation between data results, checklist items, and their evaluation process. I aim to incorporate this critical aspect into my future papers.
