Essay checkers and AI essay graders: what they catch, what they miss (the argument, the facts, your voice), and five questions to ask one

An essay checker finds the faults a rule can find: a comma splice, a subject that does not agree with its verb, a word used twice in a sentence, a passive where an active would do. An AI essay grader goes further and predicts a score. Neither can tell you the one thing that decides the mark: whether your thesis answers the question and your evidence supports it. Here is what the two kinds of tool catch, what they miss, a paragraph of ours run through both, and five questions to ask one before you trust it.

Two kinds of tool, and what each does

The checker works sentence by sentence. It has rules for spelling, grammar, punctuation and agreement, and most now add a language model that suggests a clearer word or a shorter sentence. It reads what is on the line. It does not know what the essay is for.

The grader works on the whole essay. It is a model trained on essays that people have already scored, and it predicts the score a person would give, often with a paragraph of feedback. What it reads well is the surface: how long the essay is, how varied the vocabulary is, how the sentence lengths vary, whether there are connectives such as "however" and "therefore" at the starts of paragraphs. What it reads badly is the argument.

Automated scoring is not new. It has been studied since the 1960s, and the research collected in Shermis and Burstein's handbook (2013) finds that on the features a machine can measure, its scores agree with human raters about as well as two human raters agree with each other. The critique is as old as the agreement. Perelman (2014) argued that the engines count proxies for good writing, essay length above all, and that in a timed test essay the length predicts the score closely enough that a longer, worse essay can outscore a shorter, better one. Both are true at once: the machine is consistent about the things it measures, and the things it measures are not the argument. The four-part test for any study tool starts from the same question: does it read what matters, or what is easy to read?

What a checker catches well

Give the checker its due, because the method at the end keeps it.

• The four sentence faults. The comma splice (two sentences joined by a comma), the run-on (two joined by nothing), the fragment (a piece of a sentence standing alone), and the passive that hides who did the thing. The four sentence faults explains each one and how to fix it by hand; a checker finds the first three reliably and flags the fourth.
• Agreement. A singular subject with a plural verb, usually when a phrase sits between them: "the list of reasons are".
• Articles and small words. "A" before a vowel sound, a missing "the", a dropped preposition. For a student writing in a second language this alone is worth the run.
• Repeated words. The same word twice in a sentence, or the same sentence opener four times in a paragraph.
• The real-word typo. "Their" for "there", "affect" for "effect", "defiantly" for "definitely". A spell check misses these; a checker that reads the sentence catches most of them.

That is real help, and none of it changes what your essay says.

What both miss

The argument. A thesis is a claim someone could disagree with, and it has to answer the question you were set. A thesis that argues is the test; neither kind of tool runs it, because neither reads the prompt unless you paste it in, and even then a grader scores the style of the sentence, not whether it answers. The same goes for each paragraph: whether its first sentence serves the thesis, which is the whole point of topic sentences that serve the thesis, is invisible to a tool reading sentence by sentence.

The facts. A tool can tell you a sentence is grammatical. It cannot tell you it is true. "Studies show" with no study is a grammatical sentence, and a grader will often praise it for sounding authoritative. A made-up statistic sails through.

Your voice. A checker's rewrite "improves" every sentence toward the same middle: a little longer, a little more formal, the same connectives. Accept them all and your essay sounds like every other essay that went through the same tool. A teacher who has read your work all year hears it on the first page.

The rubric. Your teacher is marking this assignment against this rubric: so much for the thesis, so much for the use of the text, so much for the mechanics. A grader predicts a general score against a general idea of a good essay. The two can disagree by a full grade.

The confidence. A grader built on a language model writes its answer fresh each time, so the same essay pasted in twice can come back with two different scores; question 5 below is how you find out for the tool you use. And a wrong suggestion arrives with the same confidence as a right one. The tool does not tell you which of its suggestions to doubt.

The integrity line

Using a checker to find your own faults, and fixing them yourself, is editing. Every writer edits, and most teachers allow a grammar checker for exactly this. Pasting an essay in and submitting the rewrite is submitting writing that is not yours, and most courses treat it as they treat any other ghost-written work. The line between the two is drawn by your course's rule, which is written down somewhere, and reading it is the first step.

Whatever the rule, keep three things in your own hand: the thesis, the examples, and the order of the ideas. Those are the essay. Everything else is the finish.

The five questions to ask any essay checker or grader

Run these before you trust a tool with a real essay. Each takes a minute. Chapter 7 of the free playbook has the general version for any AI study tool.

1. "Which sentence is wrong, and which rule did it break?" Pass: it names the sentence and the rule (a comma splice, a subject that does not agree). Fail: it hands back a rewritten sentence with no reason, which teaches you nothing for the next essay.
2. "Does my thesis answer the question I was asked?" Paste the prompt in with the thesis. Pass: it reads the prompt and says yes, no or partly, and why. Fail: it praises the thesis's style and says nothing about the question.
3. "Is this fact true?" Ask about a claim you made up for the test. Pass: "I cannot verify that" or "I do not have a source for that". Fail: a confident yes.
4. "Mark it, don't rewrite it." Pass: it points at the problems and stops. Fail: it hands you a new essay anyway.
5. "Score the same essay twice," a day apart, without changing a word. Pass: the same score, or an honest range. Fail: two different scores, which tells you what a single score is worth.

A tool that passes all five is a tool you can use the way the method below describes. One that fails questions 2 and 3 is a grammar checker, whatever it calls itself, and should be used as one.

Worked example: one paragraph of ours through a checker and a grader

Here is a paragraph we wrote for this page, on whether high schools should start later. It has three planted faults: a comma splice in the second sentence, a first sentence that names a topic instead of making a claim, and a "studies show" with no study.

School start times are an important issue for high schools today. Many students arrive tired, they have been awake since six to catch a bus that leaves at seven. A later start would let them sleep, and sleep matters for memory and attention. Studies show students who start later get better grades. Some people object that a later start would push sports and part-time jobs into the evening, and that buses would cost more if the elementary and high school routes could no longer share drivers. These are real costs. But a school exists to teach, and a classroom of students who are awake is the first condition of teaching anything at all.

What a checker can catch. The comma splice in sentence two is a rule fault, and a checker should flag it and name it. The other two faults, the topic sentence that argues nothing and the unsupported claim, are not sentence faults, so no rule it has can see them.

What a grader can see. The paragraph has the surface a grader rewards: a counterargument, a turn with "But", varied sentence lengths. Nothing a grader measures marks down the sentence that cites no study, and a rewrite that lengthens the sentences and adds connectives would raise the features it does measure without fixing any of the three faults.

The fix, by hand. The splice becomes two sentences, as the four sentence faults page shows. The topic sentence becomes a claim someone could disagree with, which is the test on the thesis page. The "studies show" sentence goes, because we have not read a study on it, and in its place is evidence the essay could actually produce.

A later school start would raise attendance and attention more than it would cost in buses and after-school time. Many students arrive tired. They have been awake since six to catch a bus that leaves at seven. A later start would let them sleep, and sleep matters for memory and attention. Our own school's attendance records would show whether first-period absences fall after a later start, and that is the evidence this essay would need before it claims anything about grades. Some people object that a later start would push sports and part-time jobs into the evening, and that buses would cost more if the elementary and high school routes could no longer share drivers. These are real costs. But a school exists to teach, and a classroom of students who are awake is the first condition of teaching anything at all.

Check the three faults: present in the first version, absent in the second. The first version is 113 words; the second is 142, a change of 29 words, almost all of it the sentence that replaced the claim with the evidence an essay would need. A checker could have made at most one of the three changes, the splice. The other two are the writer's.

How to use one without losing your voice: the method

The order is the whole method.

1. Your own pass first. Read the draft once for the four sentence faults and once for the thesis test, with the prompt beside you. Doing this before any tool is retrieval, in the sense Roediger and Karpicke (2006) measured: the attempt to find your own faults is what teaches you to stop making them. Retrieval practice explains why the attempt matters more than the answer.
2. Then the checker. Accept only a change you can explain. If you cannot say which rule the sentence broke, leave it as it was and look the rule up.
3. Then the log. Write down each fault the checker found, in a list you keep across essays. The same three come back every time. The log is the thing that improves the next essay; the tool does not, because the tool fixes this essay and remembers nothing.
4. Never the grader's rewrite. If you use a grader at all, read its score as a check on length and variety and nothing else, and close the rewrite without reading it.
5. The last read is yours, out loud, for voice. If a sentence does not sound like something you would say, it probably was not.

What the tools catch and miss, on one screen

• A checker catches: the four sentence faults, agreement, articles, repeated words, real-word typos. Keep it for those.
• Both miss: whether the thesis answers the question, whether the facts are true, your voice, your teacher's rubric, and which of their own suggestions to doubt.
• The line: finding and fixing your own faults is editing; submitting the rewrite is not your writing. Your course's rule decides where exactly. Keep the thesis, the examples and the order of ideas in your own hand.
• The five questions: which sentence and which rule; does the thesis answer the prompt; is this invented fact true; mark it, don't rewrite it; score it twice.
• The method, in order: your own pass, then the checker (only changes you can explain), then the log, never the rewrite, and the last read out loud.