Identifying a research problem (Creswell)
Problem
- large-scale assessments (SAT, GRE, etc) focus on high reliability and validity across a very wide population
- the cost of high reliability and validity across a wide population is local context and individual differences among test-takers is minimized or eliminated using a quantitative approach (Finch & French, 2019)
- faculty in higher ed tend to emulate this quantitative approach but with instruments that lack the psychometric quality of large-scale assessments (Bennett, et al., 1992 in Black & Wiliam, 1998, p. 17-20) , and tend also to over-emphasize summative assessment approaches (assessment of learning)
- pivoting to more qualitative and relational approaches to assessment (formative assessment; assessment for learning) requires foundational changes in pedagogical approaches (assessment as learning) and is difficult to accomplish outside of formal interventions (Torrie, 1989; Shepard et al., 1994, 1996; Shepard, 1995 ; all in Black & Wiliam, 1998, p. 20)
Purpose
- identify the effects of quantitative summative assessment on learners
- identify factors that encourage faculty to pivot to qualitative formative assessment
Questions
Paper 1
- What are the challenges associated with a quantitative summative approach to assessment in higher ed?
- When is it appropriate to use a quantitative summative approach to assessment?
- Why should stakeholders in higher ed consider pivoting to more qualitative formative approaches in everyday classroom practice?
Paper 2 (faculty perceptions)
- What are the experiences of faculty who have always used quantitative summative approaches to assessment?
- What are the experiences of faculty who have always used qualitative formative approaches to assessment?
- What are the experiences of faculty who have recently pivoted to a more qualitative formative approach to assessment?
Paper 3 (learner perceptions)
- what approaches to assessment do learners prefer?