Ideas for a dissertation...

Date: June 21, 2020

Topic

Problem

Purpose

Questions

Notes

<aside> 📌 SUMMARY:

</aside>

Date: June 21, 2020

Topic

The future of assessment

Problem

<aside> 🛠 modern assessment practices in higher ed rely too much on unreliable measurements of student learning such as faculty-created and/or auto-graded assessments

</aside>

• solutions may include

  • professional testing companies and/or publishers create standardized measurements with broadly applicable reliability and validity measures

    <aside> 💡 keeping in mind that reliability is sample-dependent and never 'broadly applicable'

    </aside>

  • professional testing companies and/or publishers create algorithmically determined test items at scale and implement wide-scale automated test services

  • individual faculty are empowered to view assessment as action research, borrowing from quan, qual, and mixed methods as appropriate and ensuring that they are co-researchers in the process alongside learners

Purpose

The purpose of this research will be to identify methods of assessment which centre learners at work in the tripartite nature of the university: teaching, research, and service.

Questions

Paper 1

• how well do faculty understand the assumptions and implications of CTT and IRT?
• does knowledge of CTT/IRT change faculty assessment practice?
• Do faculty understand that reliability is sample-dependent?

Paper 2

<aside> 💡 maybe combine this with algorithm paper

</aside>

• in what ways do standardized tests impact traditionally disadvantaged populations?
• in what ways do algorithmically-generated and graded assessments impact learning? motivation? agency?
• what are the perceived/actual problems and challenges with standardized algorithmic testing?

Paper 3

• What are the critical characteristics of learning/research/service-centred assessment?
• How can faculty embed teaching, research, and service in their assessment practice?
• How can action research inform assessment practice?

Notes

• could be a 43-paper dissertation, analyzing
  • assessment practices in light of CTT and IRT
  • standardized testing services
    • algorithmically generated and graded assessment
  • assessment as learning, research, and service

Resources

• Brookhart, S. M. (2003). Developing Measurement Theory for Classroom Assessment Purposes and Uses. Educational Measurement: Issues and Practice, 22(4), 5–12. https://doi.org/10/dj7bxr
• Brown, G. T. L. (2017). The Future of Assessment as a Human and Social Endeavor: Addressing the Inconvenient Truth of Error. Frontiers in Education, 2. https://doi.org/10/gg29sw
• Brown, G. T. L., & Abdulnabi, H. H. A. (2017). Evaluating the Quality of Higher Education Instructor-Constructed Multiple-Choice Tests: Impact on Student Grades. Frontiers in Education, 2, 24. https://doi.org/10/dxd8
• McMillan, J. H. (2005). Understanding and Improving Teachers’ Classroom Assessment Decision Making: Implications for Theory and Practice. Educational Measurement: Issues and Practice, 22(4), 34–43. https://doi.org/10/d26nxv
• Moss, P. A. (2005). Reconceptualizing Validity for Classroom Assessment. Educational Measurement: Issues and Practice, 22(4), 13–25. https://doi.org/10/bkxm42
• Smith, J. K. (2003). Reconsidering Reliability in Classroom Assessment and Grading. Educational Measurement: Issues and Practice, 22(4), 26–33. https://doi.org/10/dtnvgw

<aside> 📌 SUMMARY:

</aside>

Date: July 1, 2020

Topic