Mixed Methods Research in Designing an Instrument for Consumer-Oriented Evaluation

Abstract

Background: The educational product market has been gradually shifting from primarily print to primarily digital content. Educators must make quick decisions when selecting materials that will assist students in their learning.

Purpose: Purposes of this study were to describe the application of a two-stage sequential mixed-method, mixed-model design in designing an instrument for consumer-oriented evaluation and to describe implications of using mixed methods research in developing a rubric to evaluate prekindergarten through Grade 12 digital content.

Setting: The Ohio State University, Columbus, OH.

Intervention: N/A.

Research design: A two-stage sequential mixed-method, mixed-model design.

Data collection & analysis: In Stage 1, a modified electronic Delphi survey technique was implemented with US geographically dispersed subject matter experts. In Stage 2, cross-sectional focus group interviews were conducted with local teachers, administrators, and textbook publishers.

Findings: Inclusion of multiple perspectives and viewpoints from teachers, administrators, textbook publishers, and experts on importance, clarity, and appropriateness of criteria to evaluate digital content resulted in a final version of the rubric that can be used by teachers and administrators to evaluate digital content that supports students’ learning in prekindergarten through Grade 12.