Note for readers of this dataset: this document comes from the study's working repository, which is not public. File paths and commands in it refer to that repository. The data it describes is included here: results.csv, participants.csv, cases.csv, ocr-detections.csv and line-audit.csv.

# OCR calibration: completed human review and historical automatic audit

## Human image calibration: completed

The owner checked all 30 requested original images and 304 expected lines.
In the stratified sample, automatic exact-line scoring disagreed with the human
judgment on 44 of 205 determinate lines: **21.46%**, comprising 41 false
negatives and 3 false positives. Eight unreadable lines are excluded.
Both reviewed GPT tiers had 71 accepted lines out of 71; Nano Banana had 43.
The nine separately selected diagnostic images had 75 unreadable and
16 absent lines out of 91; they are not pooled into that error estimate.

These are errors of the full automatic measurement relative to the owner's
visual judgments, including recognition, block reconstruction, assignment and
normalization. A pure recognizer character-error rate cannot be inferred from
incomplete human transcriptions. The selection, model-specific rates, complete
line audit, limitations and reproducible commands are in
[manual_calibration.md](manual_calibration.md). The raw answers are in
[manual_review.json](manual_review.json). The old 2.9% reassembly indicator
below is not recognition accuracy and is not a threshold for this measured rate.

The requested image check and numeric measurement calibration are complete.
The article must use the revised findings and the scoped human comparison;
the universal market headline has been withdrawn. Production migration
validation is a separate question.

## Historical automatic audit

The text below records what was known before the human image review.
Its statements that no image was opened and no human error estimate existed
describe that earlier audit only; they are superseded by the results above.

> **This document does not contain the recognizer's error rate, and its title
> used to say it did.** The design (section 5) asked for two things: a sample
> of twenty images checked by hand, and the recognizer's error rate published
> with the results. What section 2 below actually does is compare the
> reference text with what the recognizer read — text against text, no image
> opened — and it says so in its own first line. That can show the scoring
> code assembled and matched text correctly. It cannot show how often the
> recognizer misread a picture. **No number answering that question exists in
> this study.** The figure that does exist, 2.9% in section 1, is the
> reliability of the block *reassembly*, which is a different quantity; three
> published files quoted it as the measurer's error rate and have been
> corrected.
>
> The sample also cannot be stretched to fill the gap. It was drawn as the
> best and the worst cell of each case, so ten of its twenty cells sit at a
> character-error rate of exactly 1.000 — unreadable text, where there is
> nothing to tell apart even with the picture in front of you.

This document calibrates `study/measure_all.py` (the round-1 text-fidelity
measurer): how far its automatic OCR-based scoring can be trusted, and what it
cannot see. All figures below come from the final, corrected measurer running
against `study/results/metrics.csv` (163 rows) and the OCR geometry saved
under `study/results/ocr/` (163 files) -- **not** from re-opening any image.
The source images were deleted from this disk once OCR extraction
finished; every check in this document works from the already-extracted
geometry and text, which is how the study was designed to work.

## 1. Block-reconstruction reliability (the threshold check)

Two automatic indicators are computed for every one of the 163 successful
cells:

- **scattered-across-blocks**: the share of ground-truth lines whose words
  are all present somewhere on the slide, but never within a single
  reconstructed block. This is the check on whether block reconstruction
  itself can be trusted, and it is the indicator the study's stop condition
  was attached to before the recount (see section 5).
- **reordered_line** (informational, no threshold): the share of cells where
  some ground-truth line's words are all present within its own matched
  block, but not as one contiguous, in-order run within that block. Published
  for completeness; it does not gate anything.

Final recount over all 163 cells:

| indicator | value | threshold | result |
|---|---|---|---|
| scattered-across-blocks | 2.9% (47 / 1617 lines) | must stay under 10% | **passes** -- block reconstruction is trustworthy |
| reordered_line (within-block) | 11.0% (18 / 163 cells) | none (informational) | published, not gating |

Balance check against `runs.csv`: 163 completed + 36 capability_refusal + 4
unavailable = 203 cells, which is every round-1 cell the study planned.

## 2. Manual calibration sample (20 images)

**No eyes were used to produce this section.** Everything below compares
the ground-truth text to what the recognizer read, as text. A discrepancy
that could only be confirmed by looking at the actual picture -- for
example, text that is present on the slide but the recognizer missed it
because of low contrast -- **cannot be ruled out or confirmed here**. Where a
line is marked "missing", that only means no block was found to contain its
year; it does not distinguish "the model never drew this event" from "the
model drew it and OCR failed to read it". A human still needs to look at the
9 images flagged below before that distinction can be made. The images
themselves are gone from this disk; a reviewer would need the copy kept by
the study owner.

### Selection method

20 cells, at least 2 per case (the lowest and the highest `char_error_rate`
in that case), so both strong and weak generations are represented. To reach
20 from the required 14 (2 x 7 cases), one extra low-error and one extra
high-error cell were added from each of the three cases with the most
successful generations (D3, D10, L10 -- 27/24/24 cells respectively), on the
reasoning that those cases have the most data to spot-check. No cell is
selected twice.

### Summary of the 20 selected cells

| case | participant | frame | lines exact/total | char_error_rate | blocks found/detected |
|---|---|---|---|---|---|
| D10 | gpt25_medium | 2560x1440 | 9/10 | 0.003 | 10/10 |
| D10 | seedream_4 | 2560x1440 | 0/10 | 1.000 | 0/11 |
| D18 | gpt25_low | 2560x1440 | 16/18 | 0.058 | 17/9 |
| D18 | step1x | 1392x752 | 0/18 | 1.000 | 0/7 |
| D3 | gpt25_low | 2560x1440 | 3/3 | 0.000 | 3/3 |
| D3 | kling_v3 | 2720x1536 | 0/3 | 1.000 | 0/3 |
| I10 | gpt2_medium | 2048x1152 | 10/10 | 0.000 | 10/5 |
| I10 | step1x | 1392x752 | 0/10 | 1.000 | 0/6 |
| L10 | gpt25_low | 2560x1440 | 10/10 | 0.000 | 10/10 |
| L10 | step1x | 1392x752 | 0/10 | 1.000 | 0/1 |
| M10 | gpt2_high | 2560x1440 | 9/10 | 0.007 | 10/10 |
| M10 | qwen_edit_plus | 1360x768 | 0/10 | 1.000 | 0/1 |
| P10 | gpt25_low | 2560x1440 | 10/10 | 0.000 | 10/10 |
| P10 | step1x | 1392x752 | 0/10 | 1.000 | 0/0 |
| D3 | gpt25_medium | 2560x1440 | 3/3 | 0.000 | 3/3 |
| D3 | ideogram_v4 | 2720x1536 | 0/3 | 1.000 | 0/0 |
| D10 | gpt2_medium | 2048x1152 | 9/10 | 0.008 | 10/10 |
| D10 | qwen_edit_plus | 1360x768 | 0/10 | 1.000 | 0/1 |
| L10 | gpt2_low | 2048x1152 | 10/10 | 0.000 | 10/10 |
| L10 | qwen_edit_plus | 1360x768 | 0/10 | 1.000 | 0/5 |

### Line-by-line comparison

For each selected cell: the ground-truth title vs. the recognized title-band
text, then every ground-truth line vs. the block matched to it by year (see
`study/measure_all.py`'s actual scoring for the real one-to-one assignment;
this table's own matching is a simpler, independent re-check by year number,
used only to build this table -- it agrees with `lines_exact` on all 20
cells, see the note at the end of this section).

Note key: **exact** = matches after case/punctuation normalization; **OCR
misreading** = letters or words differ, consistent with a recognizer error
or a real model typo, not a block-assembly problem; **missing line** = no
block was found containing this line's year (see the caveat above); no
sampled cell in this table showed the two other possible notes ("word order
swapped within block", "all words present but not contiguous") -- both do
occur elsewhere in the full 163, at the rates in section 1.

#### D10 / gpt25_medium (frame 2560x1440, cer 0.003)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics в Верано | exact |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Создан первый прототип манипулятора К-1 | exact |
| 2012 — Открыта первая производственная линия | 2012 Открыта первая производственная ЛИНИЯ | exact |
| 2013 — Привлечено финансирование на 2,4 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов | exact |
| 2014 — Запущена серийная модель K-3 | 2014 Запущена серийная модель К-З | OCR misreading (letters/words differ) |
| 2016 — Получена сертификация ISO 9001 | 2016 Получена сертификация ISO 9001 | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Запущена платформа KelOS для управления роботами | exact |
| 2021 — Установлена тысячная единица техники | 2021 Установлена тысячная единица техники | exact |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2025 — Представлена флагманская модель K-7 | 2025 Представлена флагманская модель K-7 | exact |

#### D10 / seedream_4 (frame 2560x1440, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история комлании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Os Hamaklu € KOшfankИ/f Kclнлt Iion alIоз Bax₽l] | OCR misreading (letters/words differ) |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Оt(ЛштА пер лй пр аТОПТИЛ Л DywalinXVF ЖifLHtT | OCR misreading (letters/words differ) |
| 2012 — Открыта первая производственная линия | 2012 Прэл А п€wve:Io o[flidTIIC КJ Ut 5Лл IHtа Ira 2 4.#" [пшпяоъ | OCR misreading (letters/words differ) |
| 2013 — Привлечено финансирование на 2,4 млн долларов | (missing -- no block found with this year) | missing line |
| 2014 — Запущена серийная модель K-3 | 2014 O1fEТ E парая ПГОрi1 gslяZ MISULMEI Б | OCR misreading (letters/words differ) |
| 2016 — Получена сертификация ISO 9001 | 2016 ПmasQuшенв Fot 7 (J J U4<ELILIЯ ISO 901/ | OCR misreading (letters/words differ) |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Пнагь еuE Ita сер mfi if 637 #ЩДFЛЪ #-3 | OCR misreading (letters/words differ) |
| 2021 — Установлена тысячная единица техники | 2021 Уlталyagtя :btlв € ig-Ivuл енll;lчщЕ ГехНии | OCR misreading (letters/words differ) |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 5алущы TImI Яl] Пт"5 ГЭ[ K<6lOS Дт уТзк5леня роtотavk | OCR misreading (letters/words differ) |
| 2025 — Представлена флагманская модель K-7 | 2025 Зууэштоколе#а 'fllа {Пialtl Gсля #ogo/ьp] | OCR misreading (letters/words differ) |

#### D18 / gpt25_low (frame 2560x1440, cer 0.058)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics Верано 2018 Запущена платформа KelOS для управления роботами | OCR misreading (letters/words differ) |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Создан первый прототип манипулятора K-1 2019 Заключено партнёрство с сетью логистических складов | exact |
| 2011 — Получен патент на систему адаптивного захвата | [Я 2011 Получен патент на систему адаптивного захвата 2020 Осуществлён переход на модульную архитектуру K-5 | exact |
| 2012 — Открыта первая производственная линия | 2012 Открыта первая производственная ЛИНИЯ 2021 Установлена тысячная единица техНики | exact |
| 2013 — Привлечено финансирование на 2,4 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов 2022 Создано подразделение сервисных роботов | exact |
| 2014 — Запущена серийная модель K-3 | 2014 Запущена серийная модель K-3 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2015 — Выход на рынок Меритании | 2015 Выход на рынок Меритании 2024 Открыт второй завод в Ново-Керне | exact |
| 2016 — Получена сертификация ISO 9001 | ЕЯ 2016 Получена сертификация ISO 9001 2025 Представлена флагманская модель K-7 | exact |
| 2017 — Открыт исследовательский центр компании | 2017 Открыт исследовательский центр компании 2026 Получена награда ФРобот годах на выставке В Верано | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2009 Основание компании Kelvorix Dynamics Верано 2018 Запущена платформа KelOS для управления роботами | exact |
| 2019 — Заключено партнёрство с сетью логистических складов | 2010 Создан первый прототип манипулятора K-1 2019 Заключено партнёрство с сетью логистических складов | exact |
| 2020 — Осуществлён переход на модульную архитектуру K-5 | [Я 2011 Получен патент на систему адаптивного захвата 2020 Осуществлён переход на модульную архитектуру K-5 | exact |
| 2021 — Установлена тысячная единица техники | 2012 Открыта первая производственная ЛИНИЯ 2021 Установлена тысячная единица техНики | exact |
| 2022 — Создано подразделение сервисных роботов | 2013 Привлечено финансирование на 2,4 млн долларов 2022 Создано подразделение сервисных роботов | exact |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2014 Запущена серийная модель K-3 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2024 — Открыт второй завод в Ново-Керне | 2015 Выход на рынок Меритании 2024 Открыт второй завод в Ново-Керне | exact |
| 2025 — Представлена флагманская модель K-7 | ЕЯ 2016 Получена сертификация ISO 9001 2025 Представлена флагманская модель K-7 | exact |
| 2026 — Получена награда «Робот года» на выставке в Верано | 2017 Открыт исследовательский центр компании 2026 Получена награда ФРобот годах на выставке В Верано | OCR misreading (letters/words differ) |

#### D18 / step1x (frame 1392x752, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Кецнаты Лапршы`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Кепразннння К _ 2 Кепданни корданния могколупдании менинштщння дерпдания | OCR misreading (letters/words differ) |
| 2010 — Создан первый прототип манипулятора K-1 | (missing -- no block found with this year) | missing line |
| 2011 — Получен патент на систему адаптивного захвата | (missing -- no block found with this year) | missing line |
| 2012 — Открыта первая производственная линия | 2012 Кепдланннои менпаюо короттнии иорупиниыв зенданиц | OCR misreading (letters/words differ) |
| 2013 — Привлечено финансирование на 2,4 млн долларов | (missing -- no block found with this year) | missing line |
| 2014 — Запущена серийная модель K-3 | (missing -- no block found with this year) | missing line |
| 2015 — Выход на рынок Меритании | (missing -- no block found with this year) | missing line |
| 2016 — Получена сертификация ISO 9001 | 2016 Кегууланннна пориупцданая Кепдданна копошеншннна малБяти на #сфуатлычия ыкотопветшоти | OCR misreading (letters/words differ) |
| 2017 — Открыт исследовательский центр компании | 2017 Келдлакенэ хериунтонния K - 4 тепдепаатш ена Пэлрптту типа кеплпдашнии перихтштыя К-1 | OCR misreading (letters/words differ) |
| 2018 — Запущена платформа KelOS для управления роботами | (missing -- no block found with this year) | missing line |
| 2019 — Заключено партнёрство с сетью логистических складов | (missing -- no block found with this year) | missing line |
| 2020 — Осуществлён переход на модульную архитектуру K-5 | (missing -- no block found with this year) | missing line |
| 2021 — Установлена тысячная единица техники | (missing -- no block found with this year) | missing line |
| 2022 — Создано подразделение сервисных роботов | (missing -- no block found with this year) | missing line |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 Кепдлнктннння #HLTUilkHH мордяттеланног Кендазена мепдлении серукя Оепння ПОПОТК HYа юоозщддния погкакстпша на БопЫтр тПIнанказ когIТаг' Пални взепрнно | OCR misreading (letters/words differ) |
| 2024 — Открыт второй завод в Ново-Керне | (missing -- no block found with this year) | missing line |
| 2025 — Представлена флагманская модель K-7 | 2025 И в подашннетн па ияпепъанна Пк\| моxттусншя рзолагы | OCR misreading (letters/words differ) |
| 2026 — Получена награда «Робот года» на выставке в Верано | (missing -- no block found with this year) | missing line |

#### D3 / gpt25_low (frame 2560x1440, cer 0.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics в Верано | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Запущена платформа KelOS для управления роботами | exact |
| 2025 — Представлена флагманская модель K-7 | 2025 Представлена флагманская модель K-7 | exact |

#### D3 / kling_v3 (frame 2720x1536, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: Kergonicхомпании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Oiiialn Company drrdи | OCR misreading (letters/words differ) |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 = Geldeп golou KelOs | OCR misreading (letters/words differ) |
| 2025 — Представлена флагманская модель K-7 | 2025 Et*А-75 'JXfijgf 1#-7 | OCR misreading (letters/words differ) |

#### I10 / gpt2_medium (frame 2048x1152, cer 0.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics в Верано ISO 9001 2016 Получена сертификация ISO 9001 | exact |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Создан первый прототип манипулятора K-1 2018 Запущена платформа KelOS для управления роботами | exact |
| 2012 — Открыта первая производственная линия | 2012 Открыта первая производственная линия 1000 2021 Установлена тысячная единица техники | exact |
| 2013 — Привлечено финансирование на 2,4 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2014 — Запущена серийная модель K-3 | 2014 Запущена серийная модель К-3 2025 Представлена флагманская модель K-7 | exact |
| 2016 — Получена сертификация ISO 9001 | 2009 Основание компании Kelvorix Dynamics в Верано ISO 9001 2016 Получена сертификация ISO 9001 | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2010 Создан первый прототип манипулятора K-1 2018 Запущена платформа KelOS для управления роботами | exact |
| 2021 — Установлена тысячная единица техники | 2012 Открыта первая производственная линия 1000 2021 Установлена тысячная единица техники | exact |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2025 — Представлена флагманская модель K-7 | 2014 Запущена серийная модель К-3 2025 Представлена флагманская модель K-7 | exact |

#### I10 / step1x (frame 1392x752, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Keigorx Кикллмтnереми Nеpих`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Комдвавешиие сомрваннии помкании (егкаычкопни понкнлуничинIсшпя мемкеупалкоррп перано иивлавення TvIф дептурлитоми ромни | OCR misreading (letters/words differ) |
| 2010 — Создан первый прототип манипулятора K-1 | (missing -- no block found with this year) | missing line |
| 2012 — Открыта первая производственная линия | (missing -- no block found with this year) | missing line |
| 2013 — Привлечено финансирование на 2,4 млн долларов | (missing -- no block found with this year) | missing line |
| 2014 — Запущена серийная модель K-3 | 2014 Копданя петый компаапвений сорбманалнне на соранмолтьпа пМ пеня лавкерна поблянстшлплвней пеипороэниськаия пIлЫИ | OCR misreading (letters/words differ) |
| 2016 — Получена сертификация ISO 9001 | (missing -- no block found with this year) | missing line |
| 2018 — Запущена платформа KelOS для управления роботами | (missing -- no block found with this year) | missing line |
| 2021 — Установлена тысячная единица техники | 2021 Котреорнь вкшяаия иониоошкасчяя паплеена сепупыня модиения IKO 9001 папкавкена попшивсякна ппulжKuuя н8 мля делыя | OCR misreading (letters/words differ) |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 Оолрзена меунывч, паилыенканин попмиершне; пьЛв qолттавветне) ЮЛИ елупиа, перралшне рслхаи спохнна лукталени поптала | OCR misreading (letters/words differ) |
| 2025 — Представлена флагманская модель K-7 | 2025 Копллеена зопыпа МОДв:кЬмр: И? ролпяояшие перреклоксците перлучканна флдеокs плм пепмаввкена панканяг пыкканка модепы K 7 | OCR misreading (letters/words differ) |

#### L10 / gpt25_low (frame 2560x1440, cer 0.000)

Title: `Kelvorix Dynamics: Company Timeline` -> recognized title band: `Kelvorix Dynamics: Company Timeline`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Kelvorix Dynamics founded in Verano | 2009 Kelvorix Dynamics founded in Verano | exact |
| 2010 — First K-1 manipulator prototype built | 2010 First K-1 manipulator prototype built | exact |
| 2012 — First production line opened | 2012 First production line opened | exact |
| 2013 — Raised 2.4 million dollars in funding | 2013 Raised 2,4 million dollars in funding | exact |
| 2014 — K-3 serial model launched | 2014 K-3 serial model launched | exact |
| 2016 — ISO 9001 certification achieved | 2016 ISO 9001 certification achieved | exact |
| 2018 — KelOS robot management platform launched | 2018 KelOS robot management platform launched | exact |
| 2021 — One-thousandth unit installed | 2021 One-thousandth unit installed | exact |
| 2023 — Series C round of 18 million dollars closed | 2023 Series С round of 18 million dollars closed | exact |
| 2025 — K-7 flagship model unveiled | 2025 K-7 flagship model unveiled | exact |

#### L10 / step1x (frame 1392x752, cer 1.000)

Title: `Kelvorix Dynamics: Company Timeline` -> recognized title band: `Kelvorix Dynamics: 2009 K-1 €15:112 с516-5`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Kelvorix Dynamics founded in Verano | (missing -- no block found with this year) | missing line |
| 2010 — First K-1 manipulator prototype built | (missing -- no block found with this year) | missing line |
| 2012 — First production line opened | (missing -- no block found with this year) | missing line |
| 2013 — Raised 2.4 million dollars in funding | (missing -- no block found with this year) | missing line |
| 2014 — K-3 serial model launched | (missing -- no block found with this year) | missing line |
| 2016 — ISO 9001 certification achieved | (missing -- no block found with this year) | missing line |
| 2018 — KelOS robot management platform launched | (missing -- no block found with this year) | missing line |
| 2021 — One-thousandth unit installed | relvotix %tborio manpular dollars Fcltd:e and uviein froundo luit greme 2021 2a4 Is, IK alveiss riots at Kisto-3-7 serillarotoral managelfent anvIs elonduntlant of dollars closed undorm iunveilled | OCR misreading (letters/words differ) |
| 2023 — Series C round of 18 million dollars closed | (missing -- no block found with this year) | missing line |
| 2025 — K-7 flagship model unveiled | (missing -- no block found with this year) | missing line |

#### M10 / gpt2_high (frame 2560x1440, cer 0.007)

Title: `Kelvorix Dynamics: история и milestones` -> recognized title band: `Kelvorix Dynamics: история И milestones`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание Kelvorix Dynamics в г. Verano | 2009 Основание Kelvorix Dynamics В Г: Verano | exact |
| 2010 — Собран первый прототип manipulator K-1 | 2010 Собран перВый ПрОТОТИП manipulator К-1 | exact |
| 2012 — Открыта production line №1 | 2012 Открыта production line Ng1 | OCR misreading (letters/words differ) |
| 2013 — Привлечено funding $2.4M | 2013 Привлечено funding $2.4M | exact |
| 2014 — Запущена serial-модель K-3 | 2014 Запущена serial-модель К-3 | exact |
| 2016 — Получен сертификат ISO 9001 | 2016 Получен сертификат ISO 9001 | exact |
| 2018 — Запущена platform KelOS v1.0 | 2018 Запущена platform KelOS v1.0 | exact |
| 2021 — Установлена 1000-я unit техники | 2021 Установлена 1000-я unit ТеХНИКИ | exact |
| 2023 — Закрыт Series C раунд на $18M | 2023 Закрыт Series C раунд на $18M | exact |
| 2025 — Представлена flagship-модель K-7 | 2025 Представлена flagship-модель K-7 | exact |

#### M10 / qwen_edit_plus (frame 1360x768, cer 1.000)

Title: `Kelvorix Dynamics: история и milestones` -> recognized title band: `Kelvorix Dynamics: история и milestones`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание Kelvorix Dynamics в г. Verano | (missing -- no block found with this year) | missing line |
| 2010 — Собран первый прототип manipulator K-1 | (missing -- no block found with this year) | missing line |
| 2012 — Открыта production line №1 | (missing -- no block found with this year) | missing line |
| 2013 — Привлечено funding $2.4M | (missing -- no block found with this year) | missing line |
| 2014 — Запущена serial-модель K-3 | (missing -- no block found with this year) | missing line |
| 2016 — Получен сертификат ISO 9001 | (missing -- no block found with this year) | missing line |
| 2018 — Запущена platform KelOS v1.0 | (missing -- no block found with this year) | missing line |
| 2021 — Установлена 1000-я unit техники | (missing -- no block found with this year) | missing line |
| 2023 — Закрыт Series C раунд на $18M | (missing -- no block found with this year) | missing line |
| 2025 — Представлена flagship-модель K-7 | (missing -- no block found with this year) | missing line |

#### P10 / gpt25_low (frame 2560x1440, cer 0.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics в Верано | exact |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Создан первый прототип манипулятора K-1 | exact |
| 2012 — Открыта первая производственная линия | 2012 Открыта первая производственная линия | exact |
| 2013 — Привлечено финансирование на 2,4 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов | exact |
| 2014 — Запущена серийная модель K-3 | 2014 Запущена серийная модель K-3 | exact |
| 2016 — Получена сертификация ISO 9001 | Е8 2016 Получена сертификация ISO 9001 | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Запущена платформа KelOS для управления роботами | exact |
| 2021 — Установлена тысячная единица техники | 2021 Установлена тысячная единица техники | exact |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 Закрыт раунд Series С на 18 млн долларов | exact |
| 2025 — Представлена флагманская модель K-7 | 2025 Представлена флагманская модель K-7 | exact |

#### P10 / step1x (frame 1392x752, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | (missing -- no block found with this year) | missing line |
| 2010 — Создан первый прототип манипулятора K-1 | (missing -- no block found with this year) | missing line |
| 2012 — Открыта первая производственная линия | (missing -- no block found with this year) | missing line |
| 2013 — Привлечено финансирование на 2,4 млн долларов | (missing -- no block found with this year) | missing line |
| 2014 — Запущена серийная модель K-3 | (missing -- no block found with this year) | missing line |
| 2016 — Получена сертификация ISO 9001 | (missing -- no block found with this year) | missing line |
| 2018 — Запущена платформа KelOS для управления роботами | (missing -- no block found with this year) | missing line |
| 2021 — Установлена тысячная единица техники | (missing -- no block found with this year) | missing line |
| 2023 — Закрыт раунд Series C на 18 млн долларов | (missing -- no block found with this year) | missing line |
| 2025 — Представлена флагманская модель K-7 | (missing -- no block found with this year) | missing line |

#### D3 / gpt25_medium (frame 2560x1440, cer 0.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | 2009 Основание компании Kelvorix Dynamics в Верано | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Запущена платформа KelOS для управления роботами | exact |
| 2025 — Представлена флагманская модель K-7 | 2025 Представлена флагманская модель K-7 | exact |

#### D3 / ideogram_v4 (frame 2720x1536, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Кеглроiх П-ф-аl педнораоене нкани язтим`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | (missing -- no block found with this year) | missing line |
| 2018 — Запущена платформа KelOS для управления роботами | (missing -- no block found with this year) | missing line |
| 2025 — Представлена флагманская модель K-7 | (missing -- no block found with this year) | missing line |

#### D10 / gpt2_medium (frame 2048x1152, cer 0.008)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | #а 2009 Основание компании Kelvorix Dynamics Верано | OCR misreading (letters/words differ) |
| 2010 — Создан первый прототип манипулятора K-1 | 2010 Создан первый прототип манипулятора K-1 | exact |
| 2012 — Открыта первая производственная линия | 2012 Открыта первая производственная линия | exact |
| 2013 — Привлечено финансирование на 2,4 млн долларов | 2013 Привлечено финансирование на 2,4 млн долларов | exact |
| 2014 — Запущена серийная модель K-3 | 2014 Запущена серийная модель K-3 | exact |
| 2016 — Получена сертификация ISO 9001 | [R 2016 Получена сертификация ISO 9001 | exact |
| 2018 — Запущена платформа KelOS для управления роботами | 2018 Запущена платформа KelOS для управления роботами | exact |
| 2021 — Установлена тысячная единица техники | 2021 Установлена тысячная единица техники | exact |
| 2023 — Закрыт раунд Series C на 18 млн долларов | 2023 Закрыт раунд Series C на 18 млн долларов | exact |
| 2025 — Представлена флагманская модель K-7 | 2025 Представлена флагманская модель K-7 | exact |

#### D10 / qwen_edit_plus (frame 1360x768, cer 1.000)

Title: `Kelvorix Dynamics: история компании` -> recognized title band: `Kelvorix Dynamics: история компании`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Основание компании Kelvorix Dynamics в Верано | (missing -- no block found with this year) | missing line |
| 2010 — Создан первый прототип манипулятора K-1 | (missing -- no block found with this year) | missing line |
| 2012 — Открыта первая производственная линия | (missing -- no block found with this year) | missing line |
| 2013 — Привлечено финансирование на 2,4 млн долларов | (missing -- no block found with this year) | missing line |
| 2014 — Запущена серийная модель K-3 | (missing -- no block found with this year) | missing line |
| 2016 — Получена сертификация ISO 9001 | (missing -- no block found with this year) | missing line |
| 2018 — Запущена платформа KelOS для управления роботами | (missing -- no block found with this year) | missing line |
| 2021 — Установлена тысячная единица техники | (missing -- no block found with this year) | missing line |
| 2023 — Закрыт раунд Series C на 18 млн долларов | (missing -- no block found with this year) | missing line |
| 2025 — Представлена флагманская модель K-7 | (missing -- no block found with this year) | missing line |

#### L10 / gpt2_low (frame 2048x1152, cer 0.000)

Title: `Kelvorix Dynamics: Company Timeline` -> recognized title band: `Kelvorix Dynamics: Company Timeline`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Kelvorix Dynamics founded in Verano | 2009 Kelvorix Dynamics founded in Verano | exact |
| 2010 — First K-1 manipulator prototype built | 2010 First K-1 manipulator prototype built | exact |
| 2012 — First production line opened | 2012 First production line opened | exact |
| 2013 — Raised 2.4 million dollars in funding | 2013 Raised 2.4 million dollars in funding | exact |
| 2014 — K-3 serial model launched | 2014 K-3 serial model launched | exact |
| 2016 — ISO 9001 certification achieved | 2016 ISO 9001 certification achieved | exact |
| 2018 — KelOS robot management platform launched | 2018 KelOS robot management platform launched | exact |
| 2021 — One-thousandth unit installed | 2021 One-thousandth unit installed | exact |
| 2023 — Series C round of 18 million dollars closed | 2023 Series C round of 18 million dollars closed | exact |
| 2025 — K-7 flagship model unveiled | 2025 K-7 flagship model unveiled | exact |

#### L10 / qwen_edit_plus (frame 1360x768, cer 1.000)

Title: `Kelvorix Dynamics: Company Timeline` -> recognized title band: `Kelvorix Dynamics: Company Timeline`

| ground truth | recognized (matched block) | note |
|---|---|---|
| 2009 — Kelvorix Dynamics founded in Verano | (missing -- no block found with this year) | missing line |
| 2010 — First K-1 manipulator prototype built | (missing -- no block found with this year) | missing line |
| 2012 — First production line opened | (missing -- no block found with this year) | missing line |
| 2013 — Raised 2.4 million dollars in funding | (missing -- no block found with this year) | missing line |
| 2014 — K-3 serial model launched | (missing -- no block found with this year) | missing line |
| 2016 — ISO 9001 certification achieved | (missing -- no block found with this year) | missing line |
| 2018 — KelOS robot management platform launched | (missing -- no block found with this year) | missing line |
| 2021 — One-thousandth unit installed | (missing -- no block found with this year) | missing line |
| 2023 — Series C round of 18 million dollars closed | (missing -- no block found with this year) | missing line |
| 2025 — K-7 flagship model unveiled | (missing -- no block found with this year) | missing line |

**Cross-check**: for every one of the 20 cells, the count of "exact" rows in
this table equals `metrics.csv`'s own `lines_exact` value for that cell
exactly (verified programmatically, 0 mismatches out of 20). This does not
prove the scoring is correct against the actual pixels (no image was
reopened to check that) -- it shows this table's independent, simpler
year-based matching agrees with the real scorer's own line-to-block
assignment on which lines it considers exact, which is the strongest check
available without eyes.

## 3. Calibration verdict

Across the 188 individual line comparisons in the 20 sampled cells: 89 exact
(47.3%), 70 missing (37.2%), 29 OCR misreadings (15.4%).

**0 of the 20 sampled cells (0%) show a discrepancy attributable to the
measurer** (title leaking into the body, blocks merged or split wrongly,
content assigned to the wrong column, or a text-level scoring mistake).
Every non-exact line in the sample is consistent with either a genuine OCR
misreading (garbled but recognizable as an attempt at the right word) or the
model simply not drawing that event legibly (the whole cell being near or at
`char_error_rate` 1.0, with mostly-garbled or absent block text throughout,
not just one isolated line).

This is evidence the measurer is currently reliable on this sample, **not**
proof it is error-free: 9 of the 20 cells contain at least one "missing"
line, and per the caveat in section 2, a low-contrast OCR miss on an
otherwise-drawn line cannot be distinguished from a genuinely absent line
without looking at the actual image. That check is left to a human with the
image copy kept by the study owner.

**So the design's section 5 requirement is not met.** It asked for a hand
check of twenty images and a published recognizer error rate. Twenty cells
were selected and compared as text; no image was looked at; no error rate was
produced. The requirement stands open, and the sample above is not a partial
answer to it — half of it is cells whose text is unreadable at
`char_error_rate` 1.000, where even the picture would settle nothing. The
material a later check would need is still here: the downscaled previews on
the viewing page and the recognition geometry under `study/results/ocr/`.

## 4. Named limitations (for the article)

- **Letter case is not checked.** Normalization lower-cases both sides
  before comparison (`study.measure.normalize_text`), same as it always did.
  A model that reproduces the right words in the wrong case (e.g. writing a
  whole line in capitals) scores identically to one that used the exact
  requested case. This is a deliberate, accepted trade-off, not an
  oversight: catching case would require a separate, much less reliable
  signal from OCR's own case detection, which is itself unreliable on small
  rendered text.

- **Block reconstruction assumes a timeline-with-years layout.** The column
  assignment in `study.blocks.group_into_blocks` anchors columns on
  whichever row of OCR text has the most year-like words (exactly 4 digits,
  starting "19" or "20"), then assigns every other word to the nearest
  anchor by horizontal position. This is correct **by construction** for
  all seven of this study's cases, which are all timelines whose columns
  each start with a year -- it is not a general infographic-layout parser.
  Applying this method to a differently-shaped infographic (no year labels,
  a non-columnar layout) would require rework, and the resulting numbers
  should not be read as evidence about arbitrary infographics, only about
  this study's own timeline template across models and languages.

- **Standalone separator dashes are not counted as words.** A "YEAR --
  text" dash is dropped during normalization (`study.measure.normalize_text`)
  because OCR routinely fails to detect its thin glyph as a separate
  character; a hyphen embedded in a token (a product code like "K-1") is
  unaffected. A model that draws the dash and a model that omits it score
  identically on this specific character.

## 5. Observation: page order does not always match task order

Not an error, and not something `score()` penalizes (it matches each
ground-truth line to its own best-fitting block regardless of where that
block sits on the page) -- but a real, separately worth-naming property of
several generations: the model does not always lay out events in the
chronological order they were given in the prompt.

Concrete example, D18 / gpt25_low (16/18 lines exact -- a near-perfect
recognition of the *text*): the recognized geometry is a 2-row-by-9-column
grid, and reading its columns left to right gives events in this year
order: 2016, 2025, 2013, 2022, 2011, 2020, 2012, 2021, 2015, 2024, 2009,
2018, 2010, 2019, 2014, 2023, 2017, 2026 -- visibly not 2009-2026 in
sequence. The model chose which pair of years shares a physical column
itself; it was not asked to keep columns chronological, and for a timeline
infographic, the visual order of events is arguably part of what "drawing a
timeline" means. This did not depress this cell's text-fidelity score (each
event's own line is still found intact), but a reader of the study should
know that "high text fidelity" does not imply "events laid out in the right
order" -- they are different properties, and only the first one is
currently measured.

Also observed when `reordered_line` was recomputed: the previous,
since-fixed version of that indicator was actually detecting this same
phenomenon (page order vs. task order) rather than genuine within-block word
disorder, which is why it is named here explicitly rather than left implicit.

## 6. Reported discrepancies

Two earlier block-reconstruction designs were tried and replaced before the
study arrived at the year-anchored assignment it uses; this is recorded here
because it bears directly on how much to trust the numbers:

- The final method was validated against the study owner's own by-hand
  table for all 24 recognized images of case D10. 21 of 24
  `char_error_rate` values matched to within rounding; three
  (`flux_2_pro`, `nano_banana`, `seedream_5_0_lite`) differ by a
  more-than-rounding amount while agreeing on `lines_exact == 0` and on
  general severity. Reported here rather than tuned to match:
  all three are cells where the model drew heavily garbled or
  near-absent text (`char_error_rate` 0.28-0.66 in this implementation,
  0.42-0.66 in the by-hand table), so the practical effect on this study's
  conclusions (which model draws legible text) is minimal, but the
  discrepancy itself is unresolved and should not be read as a perfect
  match.