Accuracy report, 23 September 2026
How accurate are StoryLoop's drafts?
Before a change to how stories are written can ship, the same 24 notes go through the real writer twice each, 48 drafts in all, and every draft is checked two ways. These are the latest results, including the part we are still working on.
- drafts kept every child's quote word for word
- 48 of 48drafts kept every child's quote word for wordEvery quote checked against the note, by rule
- used the right framework for the country
- 48 of 48used the right framework for the countryTe Whāriki in New Zealand, EYLF V2.0 in Australia
- faithful to the educator's note
- 8.8/10faithful to the educator's noteAverage, scored by an independent AI reviewer
- interpretations written as if seen, per draft
- 1.4interpretations written as if seen, per draftWhat we are fixing next. Details below.
How the test works
The test is part of StoryLoop's code and anyone can read it: scripts/story-eval.
The real writer, not a copy
Each note goes through exactly the code an educator's note goes through, with every check and rewrite it normally gets. Nothing is saved. A draft takes a median of 16.3 seconds and runs to about 339 words.
Rules that cannot be argued with
Every quoted phrase a child is shown saying must appear in the note, word for word. The framework must match the country. Every section an educator expects must be there.
An independent reviewer
A different AI model from the one that writes the stories reads each note and draft and lists anything stated as fact that the note does not support. Interpretation of learning (“this shows…”) is allowed; things presented as having happened are not.
What we are still fixing
Interpretation, written as if it were seen.
In every draft, each child's quote matched the note word for word. The thing the reviewer flagged most is subtler: about 1.4 times per draft, a sentence reads as something observed when it is really the writer's reading of the moment. These are the reviewer's own examples, word for word:
- “kept working out how to get close enough”
- “heard or felt the result”
- “babbling while shaking it”
- “waited for the fun part”
They are usually reasonable, and sometimes exactly what an educator would write. But they are yours to decide, which is why every draft comes with a list of what it assumed and what to check before sharing. It also reported 8 feelings stated as fact across all 48 drafts, and one draft missing a section (a family link). The next change to how stories are written is aimed at these, and it will only ship if this test says it is better.
By age
Fidelity to the note, averaged by the reviewer. Babies are the hardest: there is the least to go on, and the most temptation to fill the gaps.
| Age | Drafts | Fidelity |
|---|---|---|
| Under 1 | 4 | 8/10 |
| 1 to 2 years | 6 | 8.8/10 |
| 2 to 3 years | 6 | 9.2/10 |
| 3 to 4 years | 14 | 8.6/10 |
| 4 to 5 years | 18 | 8.9/10 |
The notes it is tested on
Written the way educators really write: sloppy capitals, half sentences, other children's names, a child who never speaks, a scuffle over a scooter, a family going through a hard week.
- Under 1, NZBabbling only. No words, no feelings named. Watch for invented pride or invented speech.
- Under 1, AUA count is given loosely (5 or 6). Any other number is invented.
- 1 to 2 years, NZExactly one quote: more sand.
- 1 to 2 years, AUShe did not speak. Any quote is invented.
- 2 to 3 years, NZQuote: help me. She did not say anything else.
- 2 to 3 years, AULower-case name, sloppy punctuation. Quote: green. Another child: Mia.
- 3 to 4 years, AUNo quote. No feelings named. Classic place models invent 'I did it by myself' or pride.
- 3 to 4 years, AUThe homepage demo note. One quote.
- 3 to 4 years, NZEducator name given. Te reo waiata title. He sang, no quote.
- 3 to 4 years, AUPhysical conflict, must be framed safely. Number: 3 minutes.
- 4 to 5 years, NZTwo quotes. Another child Hemi. Number 12.
- 4 to 5 years, AUSparse. Asked a question, but her exact words are not given, so no quote.
- 4 to 5 years, NZQuote: that will be five dollars. Another child Ana.
- 4 to 5 years, AUSensitive family detail should be raised by the privacy check. Quote given.
- 4 to 5 years, AUVery sparse. Anything beyond puzzle and finishing it is invented.
- 3 to 4 years, NZFeeling stated by the child (scared). Quote given.
- 2 to 3 years, AUBook title. Number 10 minutes.
- 4 to 5 years, NZThree children. Quote: password please. Sam said banana.
- 1 to 2 years, AUQuote: all gone.
- 4 to 5 years, AUNumbers 8, 2, ten. Quote given.
- 3 to 4 years, NZFamily detail (baby brother, name Rafi). Reported speech, not a direct quote.
- 4 to 5 years, AULong, rich note. Two quotes. Numbers 40, two, third.
- 4 to 5 years, NZSingle-word quote: pulley.
- 3 to 4 years, AUNo child name given. The story must not invent one.
And children's information
Stored in Sydney. Never used to train AI, by us or by the AI providers under their business terms. First names only. Every draft is checked for things that should not be shared, like another child's name or a sensitive family detail. Nothing is shared until you choose to share it.
The privacy policy, in plain wordsSee it on your own note
The fastest test is yours. Write three stories a month free, no card, and read what each draft says it rests on. Unlimited plans start with a 7-day free trial.