Problem

large-scale assessments (SAT, GRE, etc) focus on high reliability and validity across a very wide population
the cost of high reliability and validity across a wide population is local context and individual differences among test-takers is minimized or eliminated using a quantitative approach (Finch & French, 2019)
faculty in higher ed tend to emulate this quantitative approach but with instruments that lack the psychometric quality of large-scale assessments (Guskey & Link, 2019; Bennett, et al., 1992 in Black & Wiliam, 1998, p. 17-20) ,
Faculty also tend also to over-emphasize summative assessment approaches (assessment of learning) using automated technologies which perpetuate encoded bias
pivoting to more qualitative and relational approaches to assessment (formative assessment; assessment for learning) requires foundational changes in pedagogical approaches (assessment as learning) and is difficult to accomplish outside of formal interventions (Dann, 2019; Earl, 2013; Torrie, 1989; Shepard et al., 1994, 1996; Shepard, 1995 ; all in Black & Wiliam, 1998, p. 20)
- “validity of scores on educational and psycho- logical measures refers to the extent to which decisions based on these scores, and the inferences on which the decisions are based, are justified by supporting evidence (Linn, 2005).” (Finch & French, 2019, pp. 280–281)

Purpose

identify the effects of quantitative summative assessment on learners
identify factors that encourage faculty to pivot to qualitative formative assessment

What are the challenges associated with a quantitative summative approach to assessment in higher ed?
When is it appropriate to use a quantitative summative approach to assessment?
Why should stakeholders in higher ed consider pivoting to more qualitative formative approaches in everyday classroom practice?

What are the experiences of faculty who have always used quantitative summative approaches to assessment?
What are the experiences of faculty who have always used qualitative formative approaches to assessment?
What are the experiences of faculty who have recently pivoted to a more qualitative formative approach to assessment?