Home → General Knowledge Book → Important Information for 2025 → How MI Uses Validity Responses
1.4. How MI Uses Validity Responses
At Measurement Incorporated, we use multiple metrics to evaluate scoring accuracy at the group, item, and individual reader levels. One important tool for collecting these metrics is the use of validity responses. An industry-standard tool for evaluating reader accuracy, validity responses have been a key part of MI’s scoring process since the 1980s.
Validity responses are pre-scored by leadership, managers, and/or MI’s automated scoring engine, Project Essay Grade (PEG), to determine the true score for each response. These responses are included in operational scoring sets and are indistinguishable from operational student responses, providing a reliable benchmark for evaluating your ability to apply the scoring criteria accurately and consistently.
Validity reports, used by scoring leadership and managers, reflect:
- Your overall agreement with validity true scores.
- Any scoring trends you exhibit, such as a consistent tendency to assign lower (severe) or higher (lenient) scores than warranted.
- The difficulty of each validity response. Some responses clearly represent a specific score point and are easier to score accurately, while others contain characteristics of multiple score points and are more challenging. This variation is intentional, as it reflects the range of responses seen in operational scoring.
By analyzing scoring patterns across a large pool of validity responses, MI provides tailored feedback to help you monitor your scoring quality and adjust any trends that may impact accuracy. If your validity results indicate inaccurate or biased scoring, you will receive targeted feedback—both automated and from scoring leadership—to help you refine your approach.
We encourage you to use this feedback to improve your scoring by:
- Reviewing your training materials.
- Submitting line-call responses to leadership for guidance.
- Making the necessary adjustments to align with scoring expectations.
Accurate and bias-free scoring is essential to maintaining the integrity of the assessment process and ensuring that every student receives the score they have earned. Your role as a reader has a direct impact on students and their educational outcomes. We appreciate your commitment to fairness, consistency, and high-quality scoring, and we are here to support you in achieving those goals.
Thank you for your dedication to this important work.