Research paper, October 2026

Updated · 3x expanded benchmark (1,764 responses)

Within one point of TEA raters in 98.3% of cases

A 1,764-response benchmark of CoGrader’s automated STAAR constructed-response scoring across 154 TEA prompts (1,091 Short Constructed Responses and 673 Extended Constructed Responses).

PDF, 377 KB, 8 pages. Written by the CoGrader research team.

Headline results

98.3%
within one point of the official TEA score across 1,764 responses (99.5% across 2,437 traits)

99.7% within one point on SCRs; 99.4% per trait and 96.0% composite on ECRs

80.4%
exact match across all 1,764 responses (90.5% on SCRs; 75.0% per trait on ECRs)

Two trained TEA raters agree on an exact ECR trait score 63% to 75% of the time

3 sec
per response, with rubric-aligned feedback attached

Hand scoring runs 10 to 20 minutes per extended response

Executive summary

CoGrader’s grading engine landed within one score point of official Texas Education Agency (TEA) raters on 98.3% of 1,764 redesigned STAAR constructed responses (and 99.5% across all 2,437 scored rubric traits). It matched the TEA-assigned rating for the exact number of points on 80.4% of the 1,764 responses across 154 TEA prompts.

This study expands CoGrader’s initial 600-response STAAR validation nearly threefold, evaluating 1,764 student responses drawn from released TEA scoring guides and rater-training materials across grades 3 to 8, English I, English II, Biology, and U.S. History. Because the redesigned STAAR has two distinct constructed-response item types with different scoring scales and operational rules, we present results both separately and combined.

  1. Short constructed responses (SCR, 0 to 2 scale, 1,091 responses from 106 prompts)

    CoGrader matched the official TEA rating on 90.5% of all responses and fell within one point of the TEA rating 99.7% of the time (QWK = 0.92), meeting TEA’s exact-match operational standard across Reading Comprehension, Writing and Editing, Science, and Social Studies.

  2. Extended constructed responses (ECR, 0 to 5 composite scale across two traits, 673 essays from 48 prompts)

    CoGrader matched the official TEA rating on 75.0% of all traits and fell within one point of the TEA rating on 99.4% of all traits (96.0% fell within one point on the 0 to 5 composite essay score, QWK = 0.91). Two trained TEA human raters agree on an exact trait rating on 63% to 75% of operational STAAR ECRs and accept adjacent scores within one point.

  3. High-confidence routing

    In alignment with TEA’s hybrid scoring model, which allows high-confidence automated ratings to stand and borderline ratings to be flagged for human review, CoGrader achieved 97.8% exact agreement on SCRs and 85.7% per-trait exact agreement (99.1% within one point) on ECRs among the top half of responses by confidence, rising to 99.4% exact on SCRs and 94.3% per-trait exact (100% within one point) on ECRs among the top 30%.

Because CoGrader returns a rubric-aligned score and actionable written feedback in roughly three seconds rather than 10 to 20 minutes of teacher grading time, Texas districts can assign constructed-response practice at the volume students need while keeping teachers in final authority over every grade.

Does CoGrader score student writing the same way official TEA raters score it?

Why this matters: the formative assessment gap in Texas

The Texas Education Agency (TEA) changed how it assessed students as part of the 2023 STAAR redesign. TEA limited multiple-choice questions on each subject test to no more than 75% of total test points and added written constructed-response items across grade levels and subjects, resulting in six to seven times more student writing per test compared to the prior design (TEA, n.d.; Texas Tribune, 2024). To assess that volume of student writing on the spring operational exam, TEA adopted a hybrid scoring model in which a machine-based engine scores the constructed responses and sends roughly a quarter of the responses, along with lower-confidence responses, to human raters (TEA & Cambium Assessment, 2023).

Although the spring operational exam is scored by machines, classroom and benchmark writing throughout the school year are not. Full-scope STAAR Interim Assessments do not contain constructed-response items that require manual scoring, and while teachers can add SCR and ECR prompts using the Texas Formative Assessment Resource (TFAR), TFAR does not score those responses automatically. Individual teachers must manually score every practice essay, benchmark, and short answer. With each extended response taking 10 to 20 minutes to grade, a middle or high school teacher with 130 students spends over 25 hours grading a single round of benchmarks. That bottleneck leaves districts with a difficult choice: assign less writing than the redesigned STAAR demands, or return feedback weeks after students finish an assignment, too late for students to revise and for teachers to adjust instruction.

CoGrader closes that gap by bringing TEA-aligned automated scoring and rubric-based feedback into the classroom during the school year through a human-in-the-loop workflow where the teacher reviews, adjusts if needed, and approves every score.

The benchmark: separating ECR and SCR across 1,764 responses

A rigorous evaluation of STAAR automated scoring must account for how TEA constructs and grades its two constructed-response format types.

Short Constructed Responses (SCR, 0 to 2 points) are brief written responses (typically 40 to 100 words) that assess reading comprehension using text evidence, writing, revising, and editing skills, science reasoning, or social studies content accuracy. For operational STAAR scoring, TEA requires complete agreement between two raters on SCR items (TEA & Cambium Assessment, 2023, Table 3).

Extended Constructed Responses (ECR, 0 to 5 points across two traits) are multi-paragraph informational or argumentative essays written in response to a reading passage. TEA scores ECRs across two separate analytic traits: Organization and Development of Ideas (0 to 3 points) and Conventions (0 to 2 points), where a score of 0 on Organization and Development automatically results in 0 on Conventions. In operational human scoring, two independent raters score each trait. If there is adjacent agreement on a trait (a discrepancy of no more than one point), both ratings stand; otherwise, the response goes to a third reader to resolve the discrepancy. In the earlier June 2026 study of 600 responses, SCR and ECR items were reported primarily as a combined pool. In this expanded study of 1,764 officially scored redesigned STAAR responses to 154 prompts, we report both the combined benchmark and separate full-scale evaluations of the 1,091 SCR responses and 673 ECR essays across grades 3 to 5, 6 to 8, and high school (English I, English II, Biology, and U.S. History).

Exact agreement with official TEA scores vs. human rater pairs
  • CoGrader, Short Constructed Responses (SCR)

    1,091 responses across 106 prompts (0 to 2 scale)

    90.5%
  • CoGrader, all STAAR responses (pooled)

    1,764 responses across 154 prompts (81.9% across 2,437 traits)

    80.4%
  • CoGrader, Extended Constructed Responses (per trait)

    1,346 traits across 673 essays (77.7% Conventions, 72.4% Org. & Dev.)

    75%
  • Two trained TEA raters, STAAR ECR (per trait)

    Spring 2023 operational exam, by grade and trait

    63% to 75%
  • CoGrader, Extended Constructed Responses (0 to 5 composite)

    673 essays across 48 prompts (96.0% within one point)

    64.2%
  • Two trained NAEP raters, six-point writing

    Holistic 6-point scale

    57% to 67%

Sources: TEA & Cambium Assessment (2023), Table B7, pp. 37 to 38; National Center for Education Statistics, NAEP technical documentation.

Method

Each student response within the benchmark carries an official score assigned by TEA raters in published STAAR scoring guides and rater-training documents from 2022 through 2026.

To ensure that our findings reflect generalization to new assignments rather than memorization of specific prompts during calibration, calibration was restricted to an out-of-fold-by-prompt approach: when scoring students’ responses to any STAAR prompt, all responses associated with that prompt and reading passage were excluded from the calibration set. An additional, pre-declared, one-time-only evaluation was conducted on a sealed holdout group of prompts (19 ECR prompts and 42 SCR prompts) to verify performance on prompts excluded from calibration.

  • Blind to official TEA scores

    Each student response was graded from the assignment prompt, reading passage or stimulus, and TEA’s published STAAR rubrics and scoring standards, with no access to the student’s official TEA score.

  • Strict out-of-fold prompt holdout

    When scoring responses to any STAAR prompt, all responses associated with that prompt and reading passage were excluded from calibration, plus a pre-declared, one-time-only evaluation on 61 sealed holdout prompts.

  • Three complementary metrics

    We report Exact Match (TEA’s primary operational criterion for SCRs and per-trait ECR comparisons), Within One Score Point (adjacent agreement), and Quadratic Weighted Kappa (QWK).

Results

Across all 1,764 redesigned STAAR responses (154 prompts), CoGrader landed within one point of the official TEA score in 98.3% of cases and matched the official score in 80.4% of cases. Evaluated across all 2,437 individual rubric-trait scores (1,091 SCR scores plus 1,346 ECR trait scores), CoGrader achieved 81.9% exact match and 99.5% agreement within one point.

Assessment categoryPromptsEvaluated unitsScaleExact matchWithin 1 pointQWK
Short Constructed Response (SCR)1061,0910 to 2 (1 trait)90.5%99.7%0.92
Extended Constructed Response (ECR, trait level)481,346 traits (673 essays)0 to 3 (OD), 0 to 2 (CV)75.0%99.4%0.87 (OD) / 0.82 (CV)
Extended Constructed Response (ECR, composite)486730 to 5 (sum of 2 traits)64.2%96.0%0.91
All STAAR responses (response level)1541,7640 to 2 and 0 to 580.4%98.3%0.91 to 0.92
All STAAR evaluations (trait level)1542,437 traitsAll traits81.9%99.5%0.91 median

Short constructed responses (1,091 responses across 106 prompts)

On SCR items, which require exact agreement between raters according to TEA’s policy, CoGrader matched the official TEA score 90.5% of the time, agreed within one point 99.7% of the time (QWK = 0.92), and achieved 94.7% accuracy at the zero-versus-credit boundary across all four tested content areas:

SCR subject moduleResponsesExact matchWithin 1 pointQWK0-vs-credit accuracy
Social Studies and U.S. History20791.8%100.0%0.9497.6%
RLA Writing, Revising, and Editing26791.8%100.0%0.8492.1%
RLA Reading Comprehension34190.6%99.1%0.9195.0%
Science and Biology27688.0%100.0%0.9194.6%
All Short Constructed Responses1,09190.5%99.7%0.9294.7%

On the Short Constructed Responses in Social Studies and U.S. History (207 responses) and RLA Writing and Editing (267 responses), CoGrader reached 91.8% exact agreement and 100.0% agreement within one point.

In RLA Reading Comprehension (341 responses), where students must pair an implicit inference with verbatim text evidence, CoGrader scored 90.6% exact agreement and 0.91 QWK.

In Science and Biology (276 responses), CoGrader achieved 88.0% exact match, 100.0% within one point, and 0.91 QWK. On the pre-declared sealed holdout test (287 responses across 42 unseen prompts), SCR exact agreement was 89.9% (99.7% within one point, 0.91 QWK), confirming that accuracy holds on unseen prompts.

Extended constructed responses (673 essays across 48 prompts)

CoGrader was evaluated on 673 extended constructed responses (ECRs) across 48 STAAR prompts and compared with two trained TEA human raters on the operational spring 2023 exam, who agreed on the exact trait score 63% to 75% of the time depending on grade and trait (TEA & Cambium Assessment, 2023, Table B7, pp. 37 to 38). Across all 673 ECR essays (1,346 trait scores), CoGrader matched the official TEA trait score 75.0% of the time and landed within one point on 99.4% of traits:

ECR evaluation dimensionEvaluated unitsExact matchWithin 1 pointQWKHuman rater benchmark
Organization and Development of Ideas (0 to 3)67372.4%99.3%0.8763% to 75%
Conventions (0 to 2)67377.7%99.6%0.8263% to 75%
All ECR rubric traits combined1,346 traits75.0%99.4%0.8563% to 75%
ECR composite essay score (0 to 5 total)673 essays64.2%96.0%0.9157% to 67% (NAEP 6-pt)

The composite essay score is the sum of the Organization and Development and Conventions trait scores. To achieve an exact match on this composite score, the sum of both traits must match the official TEA composite score. CoGrader matched the six-point composite score on 64.2% of essays, landed within one point on 96.0% of essays (97.0% on the 268-essay sealed holdout test), achieved a 0.91 QWK, and sat at the top of NAEP’s 57% to 67% exact-agreement benchmark for two human raters on a six-point holistic writing scale (NCES, n.d.).

Confidence routing: mirroring TEA’s hybrid scoring model

CoGrader’s confidence routing mirrors TEA’s hybrid rating system. TEA uses its automated machine-scoring platform to evaluate each student response and sends lower-confidence student responses to human raters. CoGrader provides a calibration-based confidence score for each response to allow educators to use their evaluation time to assess borderline responses while quickly verifying high-confidence assessments:

Confidence tierSCR nSCR exactSCR within 1ECR nECR trait exactECR total within 1ECR QWK
All responses (100% of pool)1,09190.5%99.7%67375.0%96.0%0.91
Top 70% by confidence76397.2%100.0%47181.2%97.9%0.95
Top 50% by confidence54597.8%100.0%33685.7%99.1%0.97
Top 30% by confidence32799.4%100.0%20194.3%100.0%0.99

On the 50% of essays where CoGrader is most confident, per-trait exact agreement on ECRs rises to 85.7% (80.7% exact on the 0 to 5 composite score and 99.1% within one point, with 0.97 QWK), while SCR exact agreement reaches 97.8%. On the top 30% by confidence, SCR exact agreement reaches 99.4%, and ECR composite agreement reaches 92.5% exact and 100.0% within one point (0.99 QWK).

Scope and practical considerations

  1. Anchor and rater-training corpus

    All 1,764 responses come from TEA scoring guides and rater-training materials, which are chosen to illustrate each score point clearly. Read these figures as a ceiling relative to everyday classroom writing, which is more varied.

  2. Prompt-held-out evaluation

    Every score was produced with all responses from that prompt and reading passage excluded from calibration, and verified on 61 sealed holdout prompts. Districts evaluating CoGrader can run a blind agreement check on essays scored by their own curriculum team before adoption.

Classroom implementation and district validation

For Texas curriculum leaders and campus principals, these results show that automated classroom practice scoring can match the reliability of the state’s own human rater pool while eliminating the multi-week grading bottleneck.

  1. Teacher in the loop, always

    CoGrader does not replace the educator. Every score and rubric comment is presented as a draft for teacher review, with confidence indicators that highlight borderline papers where a teacher’s closer look adds the most value.

  2. Objective, fatigue-free consistency

    Grading 120 extended responses by hand introduces inevitable rater fatigue and halo effects between the first paper of the evening and the fortieth. CoGrader applies the exact same STAAR rubric standard to every draft.

  3. Phased district validation

    District adoption follows a structured verification protocol: an initial evidence review, a live demonstration on district-selected writing samples, and a blind agreement check against essays scored by the district’s own curriculum team, alongside FERPA, integration, and security reviews.

Appendix: benchmark breakdown by grade band

The table below summarizes performance across all three STAAR grade bands (Grades 3 to 5, Grades 6 to 8, and high school EOC) for both Extended and Short Constructed Responses. All 1,764 responses come from public TEA STAAR scoring guides and rater-training materials; no private student data from CoGrader customer districts was used in this benchmark.

Grade bandItem type & scalePromptsResponsesExact matchWithin 1 pointQWK
Grades 3 to 5ECR (0 to 5, 2 traits)2636375.9% trait / 65.6% total99.3% trait / 96.1% total0.92
Grades 6 to 8ECR (0 to 5, 2 traits)1318676.6% trait / 65.6% total99.7% trait / 97.3% total0.92
High School (English I and II)ECR (0 to 5, 2 traits)912470.2% trait / 58.1% total99.2% trait / 93.5% total0.88
Grades 3 to 5SCR (0 to 2)2927888.8%100.0%0.91
Grades 6 to 8SCR (0 to 2)4243490.6%100.0%0.92
High School (Eng. I and II, Biology, U.S. History)SCR (0 to 2)3637991.6%99.2%0.92

References

  1. Alsalem, M. S. (2024). EFL teachers’ perceptions of the use of an AI grading tool (CoGrader) in English writing assessment at Saudi universities: An activity theory perspective. Cogent Education, 11(1), Article 2430865. https://doi.org/10.1080/2331186X.2024.2430865
  2. National Center for Education Statistics. (n.d.). NAEP technical documentation: Constructed-response interrater reliability. U.S. Department of Education. https://nces.ed.gov/nationsreportcard/tdw/analysis/initial_itemscore.aspx
  3. Texas Education Agency. (n.d.). STAAR redesign. https://tea.texas.gov/student-assessment/assessment-initiatives/staar-redesign
  4. Texas Education Agency. (2022, October 20). STAAR redesign constructed response resources and updated calendar of events [To the Administrator Addressed letter]. https://tea.texas.gov/about-tea/news-and-multimedia/correspondence/taa-letters/staar-redesign-constructed-response-resources-and-updated-calendar-of-events
  5. Texas Education Agency & Cambium Assessment. (2023). STAAR hybrid scoring study, methods and results: Spring 2023 items. https://tea.texas.gov/data-reports/reports-and-studies/2023-staar-hybrid-scoring-study-2.pdf
  6. Texas Tribune. (2024, April 9). How Texas will use AI to grade this year’s STAAR tests. https://www.texastribune.org/2024/04/09/staar-artificial-intelligence-computer-grading-texas/

Cite this paper

CoGrader Research Team. (2026). Within one point of TEA raters in 98.3% of cases: A 1,764-response benchmark of CoGrader’s automated STAAR constructed-response scoring [Research paper]. CoGrader.

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305J250071 to Modern Learner Media, LLC. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

Test it on your own raters’ essays

A 30-minute discovery call to scope a blind agreement test with your assessment team.

Book a discovery call
Trusted by educators and built for compliance

Trusted by 100,000+ teachers and educators

Used at 16,000+ schools
Backed by UC Berkeley
  • SOC 2 Type 1 attested
  • FERPA Compliant
  • COPPA Compliant
Research funded by a U.S. Department of Education IES award

Frequently Asked Questions

About the STAAR scoring benchmark study.

Is this research paper peer reviewed?

No. It is a technical report by the CoGrader research team.

Why did CoGrader separate Extended Constructed Responses (ECRs) and Short Constructed Responses (SCRs) in this updated benchmark?

Because TEA scores them on different scales under different operational rules. SCRs are 0-to-2 point responses where TEA requires exact agreement between raters; across 1,091 SCRs, CoGrader matched the official TEA score 90.5% of the time (99.7% within one point). ECRs are 0-to-5 point essays scored across two analytic traits (Organization & Development 0 to 3, and Conventions 0 to 2), where TEA human raters agree 63% to 75% per trait and adjacent scores within one point stand. Across 673 ECRs, CoGrader achieved 75.0% exact agreement per trait (99.4% within one point per trait, and 96.0% within one point on the 0 to 5 composite score). Pooled across all 1,764 responses, CoGrader landed within one point in 98.3% of cases.

How does CoGrader’s accuracy compare with TEA’s own human raters?

In TEA’s Spring 2023 operational hybrid scoring report, two trained TEA human raters gave the exact same score on an ECR rubric trait 63% to 75% of the time depending on grade level and trait. CoGrader’s per-trait exact agreement across 673 ECRs is 75.0% (72.4% on Organization and Development, 77.7% on Conventions), sitting at the top of the human rater-pair range, and its composite exact agreement (64.2%) sits near the top of NAEP’s 57% to 67% benchmark for two human raters on a six-point writing scale.

Doesn’t Texas already provide automated scoring on STAAR Interim Assessments or TFAR?

No. While TEA uses hybrid automated scoring on the spring operational STAAR exam, full-scope STAAR Interim Assessments do not contain constructed-response items that require manual scoring, and the Texas Formative Assessment Resource (TFAR) does not score SCR or ECR prompts added by teachers automatically. CoGrader fills that classroom gap during the school year.

Does CoGrader replace the teacher’s score?

No. CoGrader drafts a score and rubric-aligned feedback in about three seconds. The teacher reviews it, adjusts where needed, and approves what the student sees.

Did CoGrader use private student essays from Texas school districts for this study?

No. All 1,764 responses come exclusively from publicly released TEA STAAR scoring guides and official rater-training materials from 2022 through 2026. No private student data from CoGrader customer districts was used.