Headline results
- 98.3%
- within one point of the official TEA score across 1,764 responses (99.5% across 2,437 traits)
- 80.4%
- exact match across all 1,764 responses (90.5% on SCRs; 75.0% per trait on ECRs)
- 3 sec
- per response, with rubric-aligned feedback attached
99.7% within one point on SCRs; 99.4% per trait and 96.0% composite on ECRs
Two trained TEA raters agree on an exact ECR trait score 63% to 75% of the time
Hand scoring runs 10 to 20 minutes per extended response
Executive summary
CoGrader’s grading engine landed within one score point of official Texas Education Agency (TEA) raters on 98.3% of 1,764 redesigned STAAR constructed responses (and 99.5% across all 2,437 scored rubric traits). It matched the TEA-assigned rating for the exact number of points on 80.4% of the 1,764 responses across 154 TEA prompts.
This study expands CoGrader’s initial 600-response STAAR validation nearly threefold, evaluating 1,764 student responses drawn from released TEA scoring guides and rater-training materials across grades 3 to 8, English I, English II, Biology, and U.S. History. Because the redesigned STAAR has two distinct constructed-response item types with different scoring scales and operational rules, we present results both separately and combined.
Short constructed responses (SCR, 0 to 2 scale, 1,091 responses from 106 prompts)
CoGrader matched the official TEA rating on 90.5% of all responses and fell within one point of the TEA rating 99.7% of the time (QWK = 0.92), meeting TEA’s exact-match operational standard across Reading Comprehension, Writing and Editing, Science, and Social Studies.
Extended constructed responses (ECR, 0 to 5 composite scale across two traits, 673 essays from 48 prompts)
CoGrader matched the official TEA rating on 75.0% of all traits and fell within one point of the TEA rating on 99.4% of all traits (96.0% fell within one point on the 0 to 5 composite essay score, QWK = 0.91). Two trained TEA human raters agree on an exact trait rating on 63% to 75% of operational STAAR ECRs and accept adjacent scores within one point.
High-confidence routing
In alignment with TEA’s hybrid scoring model, which allows high-confidence automated ratings to stand and borderline ratings to be flagged for human review, CoGrader achieved 97.8% exact agreement on SCRs and 85.7% per-trait exact agreement (99.1% within one point) on ECRs among the top half of responses by confidence, rising to 99.4% exact on SCRs and 94.3% per-trait exact (100% within one point) on ECRs among the top 30%.
Because CoGrader returns a rubric-aligned score and actionable written feedback in roughly three seconds rather than 10 to 20 minutes of teacher grading time, Texas districts can assign constructed-response practice at the volume students need while keeping teachers in final authority over every grade.
Does CoGrader score student writing the same way official TEA raters score it?
Why this matters: the formative assessment gap in Texas
The Texas Education Agency (TEA) changed how it assessed students as part of the 2023 STAAR redesign. TEA limited multiple-choice questions on each subject test to no more than 75% of total test points and added written constructed-response items across grade levels and subjects, resulting in six to seven times more student writing per test compared to the prior design (TEA, n.d.; Texas Tribune, 2024). To assess that volume of student writing on the spring operational exam, TEA adopted a hybrid scoring model in which a machine-based engine scores the constructed responses and sends roughly a quarter of the responses, along with lower-confidence responses, to human raters (TEA & Cambium Assessment, 2023).
Although the spring operational exam is scored by machines, classroom and benchmark writing throughout the school year are not. Full-scope STAAR Interim Assessments do not contain constructed-response items that require manual scoring, and while teachers can add SCR and ECR prompts using the Texas Formative Assessment Resource (TFAR), TFAR does not score those responses automatically. Individual teachers must manually score every practice essay, benchmark, and short answer. With each extended response taking 10 to 20 minutes to grade, a middle or high school teacher with 130 students spends over 25 hours grading a single round of benchmarks. That bottleneck leaves districts with a difficult choice: assign less writing than the redesigned STAAR demands, or return feedback weeks after students finish an assignment, too late for students to revise and for teachers to adjust instruction.
CoGrader closes that gap by bringing TEA-aligned automated scoring and rubric-based feedback into the classroom during the school year through a human-in-the-loop workflow where the teacher reviews, adjusts if needed, and approves every score.
The benchmark: separating ECR and SCR across 1,764 responses
A rigorous evaluation of STAAR automated scoring must account for how TEA constructs and grades its two constructed-response format types.
Short Constructed Responses (SCR, 0 to 2 points) are brief written responses (typically 40 to 100 words) that assess reading comprehension using text evidence, writing, revising, and editing skills, science reasoning, or social studies content accuracy. For operational STAAR scoring, TEA requires complete agreement between two raters on SCR items (TEA & Cambium Assessment, 2023, Table 3).
Extended Constructed Responses (ECR, 0 to 5 points across two traits) are multi-paragraph informational or argumentative essays written in response to a reading passage. TEA scores ECRs across two separate analytic traits: Organization and Development of Ideas (0 to 3 points) and Conventions (0 to 2 points), where a score of 0 on Organization and Development automatically results in 0 on Conventions. In operational human scoring, two independent raters score each trait. If there is adjacent agreement on a trait (a discrepancy of no more than one point), both ratings stand; otherwise, the response goes to a third reader to resolve the discrepancy. In the earlier June 2026 study of 600 responses, SCR and ECR items were reported primarily as a combined pool. In this expanded study of 1,764 officially scored redesigned STAAR responses to 154 prompts, we report both the combined benchmark and separate full-scale evaluations of the 1,091 SCR responses and 673 ECR essays across grades 3 to 5, 6 to 8, and high school (English I, English II, Biology, and U.S. History).
CoGrader, Short Constructed Responses (SCR)
1,091 responses across 106 prompts (0 to 2 scale)
90.5%CoGrader, all STAAR responses (pooled)
1,764 responses across 154 prompts (81.9% across 2,437 traits)
80.4%CoGrader, Extended Constructed Responses (per trait)
1,346 traits across 673 essays (77.7% Conventions, 72.4% Org. & Dev.)
75%Two trained TEA raters, STAAR ECR (per trait)
Spring 2023 operational exam, by grade and trait
63% to 75%CoGrader, Extended Constructed Responses (0 to 5 composite)
673 essays across 48 prompts (96.0% within one point)
64.2%Two trained NAEP raters, six-point writing
Holistic 6-point scale
57% to 67%
Sources: TEA & Cambium Assessment (2023), Table B7, pp. 37 to 38; National Center for Education Statistics, NAEP technical documentation.
Method
Each student response within the benchmark carries an official score assigned by TEA raters in published STAAR scoring guides and rater-training documents from 2022 through 2026.
To ensure that our findings reflect generalization to new assignments rather than memorization of specific prompts during calibration, calibration was restricted to an out-of-fold-by-prompt approach: when scoring students’ responses to any STAAR prompt, all responses associated with that prompt and reading passage were excluded from the calibration set. An additional, pre-declared, one-time-only evaluation was conducted on a sealed holdout group of prompts (19 ECR prompts and 42 SCR prompts) to verify performance on prompts excluded from calibration.
Blind to official TEA scores
Each student response was graded from the assignment prompt, reading passage or stimulus, and TEA’s published STAAR rubrics and scoring standards, with no access to the student’s official TEA score.
Strict out-of-fold prompt holdout
When scoring responses to any STAAR prompt, all responses associated with that prompt and reading passage were excluded from calibration, plus a pre-declared, one-time-only evaluation on 61 sealed holdout prompts.
Three complementary metrics
We report Exact Match (TEA’s primary operational criterion for SCRs and per-trait ECR comparisons), Within One Score Point (adjacent agreement), and Quadratic Weighted Kappa (QWK).
Results
Across all 1,764 redesigned STAAR responses (154 prompts), CoGrader landed within one point of the official TEA score in 98.3% of cases and matched the official score in 80.4% of cases. Evaluated across all 2,437 individual rubric-trait scores (1,091 SCR scores plus 1,346 ECR trait scores), CoGrader achieved 81.9% exact match and 99.5% agreement within one point.
| Assessment category | Prompts | Evaluated units | Scale | Exact match | Within 1 point | QWK |
|---|---|---|---|---|---|---|
| Short Constructed Response (SCR) | 106 | 1,091 | 0 to 2 (1 trait) | 90.5% | 99.7% | 0.92 |
| Extended Constructed Response (ECR, trait level) | 48 | 1,346 traits (673 essays) | 0 to 3 (OD), 0 to 2 (CV) | 75.0% | 99.4% | 0.87 (OD) / 0.82 (CV) |
| Extended Constructed Response (ECR, composite) | 48 | 673 | 0 to 5 (sum of 2 traits) | 64.2% | 96.0% | 0.91 |
| All STAAR responses (response level) | 154 | 1,764 | 0 to 2 and 0 to 5 | 80.4% | 98.3% | 0.91 to 0.92 |
| All STAAR evaluations (trait level) | 154 | 2,437 traits | All traits | 81.9% | 99.5% | 0.91 median |
Short constructed responses (1,091 responses across 106 prompts)
On SCR items, which require exact agreement between raters according to TEA’s policy, CoGrader matched the official TEA score 90.5% of the time, agreed within one point 99.7% of the time (QWK = 0.92), and achieved 94.7% accuracy at the zero-versus-credit boundary across all four tested content areas:
| SCR subject module | Responses | Exact match | Within 1 point | QWK | 0-vs-credit accuracy |
|---|---|---|---|---|---|
| Social Studies and U.S. History | 207 | 91.8% | 100.0% | 0.94 | 97.6% |
| RLA Writing, Revising, and Editing | 267 | 91.8% | 100.0% | 0.84 | 92.1% |
| RLA Reading Comprehension | 341 | 90.6% | 99.1% | 0.91 | 95.0% |
| Science and Biology | 276 | 88.0% | 100.0% | 0.91 | 94.6% |
| All Short Constructed Responses | 1,091 | 90.5% | 99.7% | 0.92 | 94.7% |
On the Short Constructed Responses in Social Studies and U.S. History (207 responses) and RLA Writing and Editing (267 responses), CoGrader reached 91.8% exact agreement and 100.0% agreement within one point.
In RLA Reading Comprehension (341 responses), where students must pair an implicit inference with verbatim text evidence, CoGrader scored 90.6% exact agreement and 0.91 QWK.
In Science and Biology (276 responses), CoGrader achieved 88.0% exact match, 100.0% within one point, and 0.91 QWK. On the pre-declared sealed holdout test (287 responses across 42 unseen prompts), SCR exact agreement was 89.9% (99.7% within one point, 0.91 QWK), confirming that accuracy holds on unseen prompts.
Extended constructed responses (673 essays across 48 prompts)
CoGrader was evaluated on 673 extended constructed responses (ECRs) across 48 STAAR prompts and compared with two trained TEA human raters on the operational spring 2023 exam, who agreed on the exact trait score 63% to 75% of the time depending on grade and trait (TEA & Cambium Assessment, 2023, Table B7, pp. 37 to 38). Across all 673 ECR essays (1,346 trait scores), CoGrader matched the official TEA trait score 75.0% of the time and landed within one point on 99.4% of traits:
| ECR evaluation dimension | Evaluated units | Exact match | Within 1 point | QWK | Human rater benchmark |
|---|---|---|---|---|---|
| Organization and Development of Ideas (0 to 3) | 673 | 72.4% | 99.3% | 0.87 | 63% to 75% |
| Conventions (0 to 2) | 673 | 77.7% | 99.6% | 0.82 | 63% to 75% |
| All ECR rubric traits combined | 1,346 traits | 75.0% | 99.4% | 0.85 | 63% to 75% |
| ECR composite essay score (0 to 5 total) | 673 essays | 64.2% | 96.0% | 0.91 | 57% to 67% (NAEP 6-pt) |
The composite essay score is the sum of the Organization and Development and Conventions trait scores. To achieve an exact match on this composite score, the sum of both traits must match the official TEA composite score. CoGrader matched the six-point composite score on 64.2% of essays, landed within one point on 96.0% of essays (97.0% on the 268-essay sealed holdout test), achieved a 0.91 QWK, and sat at the top of NAEP’s 57% to 67% exact-agreement benchmark for two human raters on a six-point holistic writing scale (NCES, n.d.).
Confidence routing: mirroring TEA’s hybrid scoring model
CoGrader’s confidence routing mirrors TEA’s hybrid rating system. TEA uses its automated machine-scoring platform to evaluate each student response and sends lower-confidence student responses to human raters. CoGrader provides a calibration-based confidence score for each response to allow educators to use their evaluation time to assess borderline responses while quickly verifying high-confidence assessments:
| Confidence tier | SCR n | SCR exact | SCR within 1 | ECR n | ECR trait exact | ECR total within 1 | ECR QWK |
|---|---|---|---|---|---|---|---|
| All responses (100% of pool) | 1,091 | 90.5% | 99.7% | 673 | 75.0% | 96.0% | 0.91 |
| Top 70% by confidence | 763 | 97.2% | 100.0% | 471 | 81.2% | 97.9% | 0.95 |
| Top 50% by confidence | 545 | 97.8% | 100.0% | 336 | 85.7% | 99.1% | 0.97 |
| Top 30% by confidence | 327 | 99.4% | 100.0% | 201 | 94.3% | 100.0% | 0.99 |
On the 50% of essays where CoGrader is most confident, per-trait exact agreement on ECRs rises to 85.7% (80.7% exact on the 0 to 5 composite score and 99.1% within one point, with 0.97 QWK), while SCR exact agreement reaches 97.8%. On the top 30% by confidence, SCR exact agreement reaches 99.4%, and ECR composite agreement reaches 92.5% exact and 100.0% within one point (0.99 QWK).
Scope and practical considerations
Anchor and rater-training corpus
All 1,764 responses come from TEA scoring guides and rater-training materials, which are chosen to illustrate each score point clearly. Read these figures as a ceiling relative to everyday classroom writing, which is more varied.
Prompt-held-out evaluation
Every score was produced with all responses from that prompt and reading passage excluded from calibration, and verified on 61 sealed holdout prompts. Districts evaluating CoGrader can run a blind agreement check on essays scored by their own curriculum team before adoption.
Classroom implementation and district validation
For Texas curriculum leaders and campus principals, these results show that automated classroom practice scoring can match the reliability of the state’s own human rater pool while eliminating the multi-week grading bottleneck.
Teacher in the loop, always
CoGrader does not replace the educator. Every score and rubric comment is presented as a draft for teacher review, with confidence indicators that highlight borderline papers where a teacher’s closer look adds the most value.
Objective, fatigue-free consistency
Grading 120 extended responses by hand introduces inevitable rater fatigue and halo effects between the first paper of the evening and the fortieth. CoGrader applies the exact same STAAR rubric standard to every draft.
Phased district validation
District adoption follows a structured verification protocol: an initial evidence review, a live demonstration on district-selected writing samples, and a blind agreement check against essays scored by the district’s own curriculum team, alongside FERPA, integration, and security reviews.
Appendix: benchmark breakdown by grade band
The table below summarizes performance across all three STAAR grade bands (Grades 3 to 5, Grades 6 to 8, and high school EOC) for both Extended and Short Constructed Responses. All 1,764 responses come from public TEA STAAR scoring guides and rater-training materials; no private student data from CoGrader customer districts was used in this benchmark.
| Grade band | Item type & scale | Prompts | Responses | Exact match | Within 1 point | QWK |
|---|---|---|---|---|---|---|
| Grades 3 to 5 | ECR (0 to 5, 2 traits) | 26 | 363 | 75.9% trait / 65.6% total | 99.3% trait / 96.1% total | 0.92 |
| Grades 6 to 8 | ECR (0 to 5, 2 traits) | 13 | 186 | 76.6% trait / 65.6% total | 99.7% trait / 97.3% total | 0.92 |
| High School (English I and II) | ECR (0 to 5, 2 traits) | 9 | 124 | 70.2% trait / 58.1% total | 99.2% trait / 93.5% total | 0.88 |
| Grades 3 to 5 | SCR (0 to 2) | 29 | 278 | 88.8% | 100.0% | 0.91 |
| Grades 6 to 8 | SCR (0 to 2) | 42 | 434 | 90.6% | 100.0% | 0.92 |
| High School (Eng. I and II, Biology, U.S. History) | SCR (0 to 2) | 36 | 379 | 91.6% | 99.2% | 0.92 |
References
- Alsalem, M. S. (2024). EFL teachers’ perceptions of the use of an AI grading tool (CoGrader) in English writing assessment at Saudi universities: An activity theory perspective. Cogent Education, 11(1), Article 2430865. https://doi.org/10.1080/2331186X.2024.2430865
- National Center for Education Statistics. (n.d.). NAEP technical documentation: Constructed-response interrater reliability. U.S. Department of Education. https://nces.ed.gov/nationsreportcard/tdw/analysis/initial_itemscore.aspx
- Texas Education Agency. (n.d.). STAAR redesign. https://tea.texas.gov/student-assessment/assessment-initiatives/staar-redesign
- Texas Education Agency. (2022, October 20). STAAR redesign constructed response resources and updated calendar of events [To the Administrator Addressed letter]. https://tea.texas.gov/about-tea/news-and-multimedia/correspondence/taa-letters/staar-redesign-constructed-response-resources-and-updated-calendar-of-events
- Texas Education Agency & Cambium Assessment. (2023). STAAR hybrid scoring study, methods and results: Spring 2023 items. https://tea.texas.gov/data-reports/reports-and-studies/2023-staar-hybrid-scoring-study-2.pdf
- Texas Tribune. (2024, April 9). How Texas will use AI to grade this year’s STAAR tests. https://www.texastribune.org/2024/04/09/staar-artificial-intelligence-computer-grading-texas/
Cite this paper
CoGrader Research Team. (2026). Within one point of TEA raters in 98.3% of cases: A 1,764-response benchmark of CoGrader’s automated STAAR constructed-response scoring [Research paper]. CoGrader.
The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305J250071 to Modern Learner Media, LLC. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.
Test it on your own raters’ essays
A 30-minute discovery call to scope a blind agreement test with your assessment team.
