how accurate is a sheet music scanner,
measured properly.

Summary

Sheetly scored 96.44% strict F1 on 31 clean scores, 95.28% on the same 31 degraded to simulated phone photographs, and 90.3% mean / 99.2% median on 60 systems cut from real scanned pages in the public OLiMPiC suite. Audiveris 5.11.0 scored 88.43%, 50.43% and 57.6% / 64.4% on the same pages, or 69.2% / 74.4% if its ten failures are excluded rather than scored zero. oemer scored 38.45% and 9.81% on the 31-piece suite.

None of that has to be taken on trust. The ground truth, every system’s raw MusicXML output, the scorer and the photograph degradation are published as a 2 MB download; one command recomputes the two 31-piece tables in well under a minute, on a laptop, with no dependencies. The OLiMPiC table needs one extra step, because that dataset’s ground truth is not ours to redistribute. If a figure here were flattering, the files would say so. Check it yourself →

Every system was measured on the same pinned input files with the same scorer: Sheetly’s August 2026 production pipeline, Audiveris 5.11.0 and oemer 0.1.x. Both are the current releases of those projects, and unchanged since they were run. This is a dated snapshot rather than a permanent property of any of the three, and it is written by the people who make one of them. That is exactly why the inputs, the outputs and the scorer are published instead of described.

Why a benchmark

Every sheet music scanner describes itself as accurate, and almost none says accurate at what, measured how, on which kind of page. A benchmark replaces that sentence with three things that can be argued with: a metric, a set of inputs, and a procedure that treats every system identically. Without those, a comparison is an opinion with a percentage sign on it.

The particular question we wanted answered is not “can OMR read a flatbed scan”, which it can, and has for years, but “what happens when the page is a phone photograph, with perspective, shadow and curl”. That is the input a player in a practice room actually has, and it is where published accuracy figures, which are almost always measured on clean input, stop describing the experience.

The second question is whether a system's own suite can be trusted. It cannot, entirely, which is why the third suite on this page is one we did not author.

What is measured

Every figure on this page is strict F1 at the note level. A transcribed note is counted as correct only when its pitch, its onset within its bar and its duration all match the ground truth. F1 is the harmonic mean of precision, the fraction of transcribed notes that were right, and recall, the fraction of true notes that were found. A score of 100% is a perfect transcription; 0% is no usable notes at all.

How notes are matched

Ground truth and transcription are each flattened to a sequence of notes ordered by onset, and the two sequences are aligned by longest common subsequence under one predicate: same pitch, same onset within the bar to a tolerance of a sixty-fourth, same duration to the same tolerance. Extra notes in the transcription cost precision; missing notes cost recall.

Onset is measured inside its bar rather than from the start of the piece, deliberately: a system that drops a rest in bar 3 should lose that bar, not every bar after it. The bar a note lands in is not part of the predicate either, only the order the alignment must respect. This is a real relaxation and it is worth saying who it helps. It is not us. Re-scored with the bar number in the predicate, Sheetly falls 0.0 and 2.1 points on the two suites, Audiveris 1.9 and 13.6, oemer 16.6 and 3.8. A stricter metric than the one published would widen every gap on this page.

Why pitch-only accuracy flatters

Most quoted OMR accuracy figures count pitches. A transcription that gets every pitch right and every rhythm wrong scores close to 100% on a pitch measure and is, to a player, unusable: it will play back as a different piece. Strict F1 scores that transcription as the failure it is. It also punishes the common partial failures that pitch-only measures either ignore or count once: a dropped dot, a tuplet read as straight quavers, a missed clef change that shifts a passage by an octave.

The consequence is that strict F1 numbers look lower than the figures scanners usually quote. That is the point. A 98% here means roughly one note in fifty is wrong in some respect; a 98% on a pitch-only measure can hide a rhythmically broken page.

What the metric does not measure

“Strict” is strict about three things and silent about the rest, and the silences should be named. Scored: pitch, onset and duration, for every pitched note. Not scored: rests, which staff or hand a note was assigned to, enharmonic spelling (G♯ against A♭), clefs, key signatures, time signatures, repeats and voltas, slurs, articulations, dynamics and lyrics. A transcription that puts the whole left hand on the right staff, or spells every accidental the wrong way, loses nothing here. The scorer computes staff accuracy, spelling accuracy and an exact-measure rate as well, published per piece in the results JSON as staff_acc, spell_acc and measure_exact, but the headline figure is note-level only, and it applies identically to every system in the table.

The established metric: OMR-NED

After this page was first published, the maker of a competing engine (Soundslice) made two public criticisms of it: strict F1 scores only pitched notes and ignores the rest of the notation, and the scorer is our own code when established metrics exist. Both points are fair, so as of August 31 the page reports the established whole-notation metric alongside strict F1: OMR-NED, the OMR normalized edit distance introduced with the Sheet Music Benchmark at ISMIR 2025 (arXiv:2506.10488) as the successor to the older symbol error rate. It counts edit operations over every notation symbol, normalized by the symbol count of both scores: note heads, beams, dots, accidentals, ties, slurs, articulations, ornaments, clefs, key and time signatures, dynamics, directions and lyrics. 0.0 is a perfect transcription and lower is better. None of the scoring code is ours this time: every comparison is musicdiff 5.2, the metric authors’ own tool, at default settings, over the same published page-one files strict F1 is scored on.

The result is worth stating against our own interest: on clean renders, Audiveris wins this metric, 0.216 to Sheetly’s 0.236 mean and 0.260 to 0.265 corpus-level, precisely because it reads slurs, articulations, dynamics and ornaments that Sheetly does not read at all. Which marks those are is published one by one on the symbols page, and this table is what those gaps cost when the whole notation is scored. On phone photographs, the input the product exists for, Sheetly leads under both metrics, 0.260 against 0.538. Strict F1 stays as the headline because it answers the player’s question of whether they will hear the right notes, while OMR-NED answers the researcher’s: how much of the notation survived. Which of the two describes your experience depends on the job. The marks Sheetly does not read are dynamics, slurs and ornaments. They say how a page should be interpreted, not which notes sound, so they cost little when the job is photographing a part and practising against it, and a lot when the job is re-engraving or archiving a score with every marking intact. For that second job, on a clean flatbed scan, this table says to use Audiveris, and tools built for notation work, which is a different product category from a practice app. A fair comparison needs both numbers, which is why both are now published, per piece, with the code: omrned-results.json and omr_ned.py.

The three suites

Three sets of inputs were used, each answering a different question. The first two are ours; the third is not, and is the one to trust most.

1. Thirty-one clean renders

Thirty-one pieces chosen to span the cases that break scanners: single-line parts, grand-staff piano writing with independent voices and cross-staff beams, multi-part vocal and chamber scores, repeats and voltas, tuplets, dense ledger lines, and unusual metres and clefs. Each has ground-truth notation and was rendered clean, as a flat, evenly lit, square-on page. This is the easiest case, and the one most published figures describe.

2. The same thirty-one as simulated phone photographs

Each clean render was passed through a deterministic degradation that imitates a phone photograph of a page on a stand: perspective skew, uneven lighting and shadow, page curl, mild blur and sensor noise, at a fixed “moderate” severity. Deterministic means the degradation is reproducible from the same source file rather than sampled anew per system. Because we designed this degradation, this suite is the weakest evidence on the page: it would be possible, in principle, to tune a system to survive exactly this and nothing else.

3. Sixty systems cut from real scanned pages (OLiMPiC)

OLiMPiC 1.0-Scanned is a public dataset from ICDAR 2024, released under CC BY-SA, which pairs crops of original IMSLP library scans with MusicXML ground truth. It is the one suite here we did not author, and the artefacts in it are genuine rather than simulated: bleed-through, skew, faded print, binding shadow. This is the check on suite 2: if a system only looked good on our own degradation, it would show here.

Two things about it should be stated plainly, because they make it both stronger and narrower evidence than the other suites.

• Each sample is one system, not a page. OLiMPiC crops per staff system, so a sample is a grand-staff system lifted out of a scanned page: 16 to 122 notes, 52 on average. That removes the page-level problems a phone photograph has: finding the staves across a whole page, grouping systems, following the music from one system to the next. A figure measured here is a figure for reading real engraved ink, not for reading a whole page.
• The 60 were not chosen by us, but the rule has a bias worth naming. The selector sorts the dataset’s own sample ids and takes them round-robin, one system per score; the first 60 are therefore always the same 60, and picking a different set would mean changing the rule rather than the result. Because 60 is one full round, every sample is the first system of page one of its score. That is the strip that carries the clef, key and time signature, and the one least likely to sit in a page interior. The list is in samples.txt; a run with --n=200 would reach further into each score.

Systems compared and how each was run

Three systems appear: Sheetly, Audiveris 5.11.0 and oemer 0.1.x. The rule for inclusion was simple. A system is in the table only if it can be run head-to-head, with identical input files in, a machine-readable score out and the same scorer applied, so that the comparison is checkable rather than reported.

• Sheetly. The production recognition pipeline, run on each input exactly as an uploaded photograph would be, with one deliberate difference: the LLM and vision passes are switched off, so every run is deterministic. Those passes read the title block and guess the instrument; they do not read notes, and every figure here is note-level. You can see the switch in our own output files, each of which carries the line LLM/vision passes are disabled in the offline pipeline. It costs us nothing on this metric and it means a re-run cannot drift.
• Audiveris 5.11.0. The open-source desktop engine, run from its command-line interface in batch mode on each input with default settings, exporting MusicXML.
• oemer 0.1.x. The open-source end-to-end neural OMR project, run from its command-line interface on each input, exporting MusicXML.

Every output was scored by the same strict F1 scorer against the same ground truth, and no system was given a friendlier page or a friendlier metric. Inputs were byte-identical on the clean suite. On the photographed suite they were not: the competitor runner was staged with page one of each piece only, so Sheetly saw second pages that Audiveris and oemer never received. That is why the published figures score page one for everybody. The scoring is like-for-like, the staging was not, and both halves of that sentence should be read together.

Why PlayScore 2 and Halbestunde are not in the table

They are the two products people actually weigh Sheetly against, and their absence is the biggest hole in this page. The reason is mechanical rather than strategic: both are phone apps with no batch interface, and in both the MusicXML export needed to score anything sits behind a paid tier. There is no way to push 62 files through either one automatically. A fair comparison needs a person with a phone, importing 62 images by hand, exporting 62 files, and correcting none of them.

So we have published everything that person would need.

• The test pages (29 MB). Page one of all 31 pieces, as clean renders and as the photographed versions: the pages every figure on this page is scored over. The second pages, which only some engines received and which nothing here is scored on, are not in it.
• The ground truth and the scorer (2 MB), so an export from any app can be scored by the same code that produced every figure here.
• The protocol: import each image, first recognition attempt only, no manual correction, export MusicXML, record the app version and the date. An app that refuses a page scores zero for that page rather than being excused from it, the same rule Audiveris and oemer were held to.

If you run it, send the exports to hello@trysheetly.com and we will publish the table with your name on the method and a link to your files, whatever it says. If Sheetly loses a column, the column goes up losing, dated, next to the rest. Until someone does that, this page makes no claim about either product, and neither should anyone quoting it.

Results

On the 31-piece suite Sheetly scored 96.44% clean and 95.28% photographed; Audiveris 88.43% and 50.43%; oemer 38.45% and 9.81%. On the 60 OLiMPiC systems Sheetly scored 90.3% mean and 99.2% median with usable output on all 60 scores; Audiveris 57.6% mean and 64.4% median with usable output on 50 of 60, or 69.2% and 74.4% over only the 50 it transcribed.

31-piece suite: clean renders and simulated photographs

OLiMPiC: 60 systems cut from real scanned pages

The ordering is the same on all three suites, and the shape of the result is that the photograph column has almost stopped costing anything: a page photographed on a stand scores just over a point below the same page rendered flat, where the best free engine loses thirty-eight. The gap to that engine is eight points on clean pages, forty-five on photographs, and thirty-three on real scans.

What changed between July and August

Fourteen changes to the recognition pipeline, not one of them a retrained model. Each is a deterministic, evidence-gated correction that only fires when it can prove its case on the page in front of it, and each had to leave a set of six reference pieces at a perfect score before it was allowed to ship. Most fix one structural misreading: a pair of staves welded into a single row, a tenor clef whose small printed 8 was being read as a lyric syllable, a time signature inferred from bar lengths instead of read off the page, a system of unequal size re-shredded rather than aligned to its neighbours. Eight of the thirty-one clean pieces improved, ten of the same pieces photographed, and five of the sixty real scans; one photographed piece slipped by three thousandths, and nothing else went backwards. Those per-piece counts come from the July run, whose per-piece figures we have not published, so they are the one set of numbers on this page you cannot check against a file. The suites, the inputs and the scorer did not change, which is what makes July and August comparable at all.

Every piece, every score

Benchmarks are usually published as three numbers, which is exactly enough to hide behind. Here is the whole thing: all 31 pieces, every system, clean and photographed, sorted by size. The pieces where Sheetly does worst are in it too. A dash means the system produced no usable transcription for that page, which is counted as zero in the means above rather than quietly dropped.

Twelve of the 31 come back note-perfect on clean renders and eleven survive the photograph untouched. Two clean pieces and four photographed ones fall below 90%, and those are where the next run’s work is. On the other side of the table, oemer produced nothing at all for eight clean pages and fourteen photographed ones, and Audiveris for three photographed ones.

The aggregate also hides five clean pieces where Audiveris beats us: the Chopin mazurka (96.6% against our 91.1%), Beach’s Prayer of a Tired Child (94.1% against 90.2%), Ryan’s jig (93.0% against 87.4%), the C. P. E. Bach (89.4% against 84.0%) and the third Schumann quartet movement (99.0% against 98.0%). The eight-point gap on clean renders is not uniform superiority; it is Sheetly being steadier, with Audiveris collapsing on a handful of pages rather than losing a little everywhere. On photographs it is behind on thirty of the thirty-one, and ties the remaining one, Aloha ‘Oe, where both are perfect.

The 60 real scans, one by one

The same again for the OLiMPiC suite, with Audiveris alongside. The distribution matters more than the mean here, so it is worth saying exactly what it is: 27 of the 60 are perfect, 47 are at 90% or better, 11 fall below 80%, and 4 fall below 50%, the worst of them at 8.9%. That tail is the whole distance between the 99.2% median and the 90.3% mean. In practical terms, most real scans come back essentially right and roughly one in fifteen comes back badly enough to be worth rescanning or fixing by hand. Anyone quoting the 90.3% without the tail, us included, would be quoting it wrong.

Multi-staff and dense scores

The most common thing said about a young scanner, usually without anyone having measured it, is that it copes with a melody line but falls apart on real polyphony: a quartet, a chorale, a piano texture with independent voices. It is a fair thing to want checked, so the suite was built with those cases in it from the start and the per-piece results are published rather than summarised.

Seven of those come back note-perfect on a clean render, including the four-voice Bach chorale, the five-voice Palestrina and the Große Fuge, which is about as dense as four staves get. Density on a single system behaves the same way: the page of Maple Leaf Rag in the suite carries 518 notes of two-hand ragtime and scores 100% clean and 99.4% photographed.

Two honest qualifications. The pieces that cost Sheetly most on this list are the ones with words: Beach’s Prayer of a Tired Child, choir over piano with lyrics under every staff, at 90.2%. Audiveris beats us there, which is in the table because leaving it out would make the table worthless. And this metric scores notes: pitch, onset and duration. It does not score dynamics, hairpins, articulation or expression marks, so nothing here is evidence about those, for any system in the table.

Reading the numbers fairly

The headline gap on photographs, 95.28% against 50.43%, is real but needs its context: Audiveris was built for flatbed scans and is measured there outside its design intent, the mean and the median tell different stories, and a single F1 figure says nothing about which notes were wrong or how much work fixing them would be.

Audiveris on photographs

Audiveris is desktop software designed around clean scans, with a correction editor for the residue. The photograph column measures it on an input its authors did not build it for. The number is accurate and it matters if your source is a phone, but on the clean column, the case Audiveris was actually made for, it scores 88.43%, which is a genuinely strong result for free software. Anyone with a flatbed scanner and a desktop should try it.

Audiveris on single-system crops

The photograph column gets a caveat about measuring Audiveris outside its design intent, and the OLiMPiC column deserves the same one. Each OLiMPiC sample is a lone staff system of 16 to 122 notes with no page around it. Audiveris is a page-level engine that expects to see a whole sheet and reason about its layout; handed a single strip, it has none of that context, which is the most likely reason it returned nothing at all for ten of the sixty and why its mean over everything (57.6%) sits well below its mean over what it managed (69.2%). The comparison is still like-for-like, since Sheetly was handed exactly the same strips, but it is not a measurement of Audiveris reading a page.

Mean against median

On OLiMPiC, Sheetly's median is 99.2% and its mean 90.3%. The distance between them is the shape of the failures: most scores come back almost entirely right, and a minority come back badly wrong, usually because the structure of the page, which is how many staves belong to which instrument, was misread and everything downstream inherited the error. The mean is the honest summary of the whole suite; the median is the honest description of what a typical page feels like. Both are reported because quoting either alone would mislead in opposite directions. Forty-seven of the sixty scores are at 90% or better and twenty-seven are perfect; the mean is held down by a tail of about a dozen pages, and closing that tail is what the next run is for.

What a 99% median means to a player

Roughly one note in a hundred wrong in pitch, onset or duration, on a typical real scan. In practice that is a page that plays correctly enough to learn from, with a bar or two that sound off and announce themselves the first time through. A 58% is a page where nearly every other note has something wrong with it, which is not a transcription so much as a starting point for manual correction.

Limitations

This benchmark is Sheetly’s own work on Sheetly’s own system, and its second suite uses simulated rather than real photographs. Publishing the files removes the question of whether the numbers are what we say they are; it does not remove the questions below, which are about what the numbers cover.

• We wrote the 31-piece suite. The pieces were chosen to be hard in the ways we thought mattered, which is also a way of choosing what not to test.
• The photographs are simulated, at the middle setting. The degradation is a model of a phone photograph, not a photograph, and the published figures use its “moderate” severity rather than its “heavy” one. Real phone shots vary more than any fixed model does, in both directions, and a bad one is closer to heavy than to moderate.
• The competitor runs are dated July. Audiveris 5.11.0 and oemer 0.1.x were measured in July 2026 and have shipped no release since, so a fresh run on these files would produce the same output. Their raw output from that run is in the bundle, which means the claim is checkable rather than asserted: re-run either engine yourself and compare it file by file.
• The photographed suite was staged short. Only page one of each piece reached the competitor runner, so the first published photograph figures charged Audiveris and oemer for pages they were never given. Everything is now scored on page one for every engine, which is like-for-like but throws away 44% of the notes: 594 bars of 935, 6,610 notes of 11,861; the multi-page files are still in the bundle behind --all-pages.
• A zero is an absent file, not a log. Where an engine produced no usable output, the bundle represents that as a missing file. That happened to oemer on 8 clean and 14 photographed pages, and to Audiveris on 3 photographed ones. The runner did capture exit codes and stderr for those pages, but they are in cloud storage we cannot currently read, so they are not in the bundle. Until they are, “the engine failed here” is our word rather than a log you can inspect, and it is the same shape as the staging defect corrected above. Both engines are open source and the input images are published, so the failures can be reproduced independently.
• It is a single run per figure. Recognition pipelines are not perfectly deterministic; the same page can decode slightly differently on different runs, and a handful of pieces in the suite are known to jitter by several points. The figures here were not averaged across repeated runs. Repeating the August run on the same pages did return byte-identical decodes, but that tests one build on one machine and is weaker than it sounds: it says the pipeline is deterministic given identical inputs, not that a re-photographed page would score the same.
• The engine moves. Sheetly's pipeline is revised continuously, and the fourteen changes between the July and August runs are the ordinary rate of it. The numbers describe a run, not a guarantee about any particular page.
• Closed products are absent. PlayScore 2 and Halbestunde, the two products most people are actually choosing between, cannot be batch-driven, so they are not here and nothing on this page ranks them. The pages and the protocol to test them are published; if you run it, we will publish what you find.
• Default settings. Audiveris and oemer were run with defaults. An expert user tuning Audiveris per page would likely do better on the clean suite than the figure shown.

Reproducing it

A benchmark you cannot recompute is a press release. Everything behind the 31-piece tables is published, so the figures can be checked by a stranger with a laptop and no reason to be kind to us.

python3 score.py .

python3 score.py clean31/ground-truth/joplin-maple-leaf.musicxml \
                 clean31/sheetly/joplin-maple-leaf/p1.musicxml

What is in the bundle

• The ground truth. The reference notation for all 31 pieces, engraved from public-domain sources.
• Every system’s output. The MusicXML that Sheetly, Audiveris 5.11.0 and oemer 0.1.x produced, page by page. A piece an engine failed on is simply absent, and scores zero.
• The scorer. This is score.py, lifted out of our evaluation harness rather than rewritten for publication. It is the code that produced every figure on this page.
• The OMR-NED runner. This is omr_ned.py, which recomputes the whole-notation metric with the reference implementation (pip install musicdiff, then python3 omr_ned.py ., about fifteen minutes). It contains no scoring logic of ours; per-piece output is in omrned-results.json.
• The manifest and per-piece results as JSON, for anyone who wants the numbers without the parsing.
• The OLiMPiC selector. This is olimpic60/select.py, the rule that chose the 60 samples, so the selection can be re-derived rather than taken on trust, and re-run with a larger n to reach past the first system of each score.
• The competitor commands. The exact CLI invocation, timeouts and failure handling for Audiveris and oemer, so “default settings, batch export” is a claim you can read rather than take.
• The photograph degradation. This is photo_sim.py, the deterministic model that made the second suite, so its severity can be inspected instead of assumed.
• Our 60 OLiMPiC transcriptions. One file per real scanned score. We cannot redistribute the OLiMPiC ground truth without putting the bundle under share-alike, but our output for every one of the sixty is here, which is the “usable output on 60 of 60” claim in a form you can count and score yourself.

The input images themselves (29 MB) are a separate download, since they are only needed to re-run a recognition engine rather than to re-score one. Both archives are listed in SHA256SUMS.txt, so a copy can be checked against the one we published. The engraved notation, the transcriptions and the two scripts are CC BY 4.0, so they are usable, including to argue against us; the OLiMPiC transcriptions inherit that dataset’s CC BY-SA.

Going further than re-scoring

For the OLiMPiC suite, the ground truth is the dataset’s own and is not ours to redistribute; olimpic60/README.txt in the bundle gives the sample list and the exact command to score our transcriptions against it once you have fetched it.

Re-scoring proves the arithmetic. To test the claim itself, run the engines again: Audiveris and oemer are open source and take an image on the command line, and the input images are published above. For the third suite, fetch OLiMPiC from its own source, which is public under CC BY-SA, and score our sixty transcriptions against its ground truth with the same score.py. Sheetly’s own recognition is a hosted service and cannot be shipped in a zip; put the same pages through the app and compare, which is the test a sceptical reader would run anyway.

The most useful reproduction is still the simplest. Take the hardest page you own, photograph it once, and put it through two or three scanners. The benchmark exists so that the result does not surprise you.

Changelog

• 2026-08-31: A second metric, adopted from the literature after public criticism. The maker of Soundslice, a competing OMR engine, pointed out on Reddit that strict F1 scores only pitched notes and that the scorer is our own code when established metrics exist. Both criticisms were fair. The page now also reports OMR-NED (arXiv:2506.10488, ISMIR 2025), computed with the metric authors’ own tool, musicdiff 5.2, at default settings over the already-published page-one files. It changes a conclusion, and the change is published: on clean renders Audiveris 5.11.0 wins the whole-notation metric, 0.216 to Sheetly’s 0.236 mean (0.260 to 0.265 corpus-level), because it reads slurs, articulations, dynamics and ornaments Sheetly does not; Sheetly leads the photograph suite 0.260 to 0.538. The runner and per-piece results were added to the bundle, and the symbols page now names exactly which marks account for the clean-render gap.
• 2026-08-23: Correction, against our own figures. An outside reading of the published bundle found that the competitor runner had only ever been given page one of each photographed piece, while the scorer charged it for the full multi-page ground truth. Everything is now scored on page one for every engine. Audiveris’s photograph figure rises from 38.55% to 50.43% and oemer’s from 7.70% to 9.81%; Sheetly’s fall from 97.03% and 96.00% to 96.44% and 95.28%. The conclusion is unchanged and the margin on photographs is forty-five points rather than fifty-seven. Earlier entries below were scored multi-page.
• 2026-08-23: Corrections to this page found by an outside audit of the bundle, listed because they were ours: the reproduction instructions still quoted the pre-correction figures; “identical inputs” was left standing after we had documented that the photographed inputs were not identical; the metric was described as matching onsets across the piece when the code matches them within the bar; the cost of page-one scoring was given as a third of the notes when it is 44%; an earlier note here said Sheetly lost five clean pieces to Audiveris “rather than three” when it loses the same five under both scoring modes; and Audiveris’s OLiMPiC median was published under the harsher of two conventions without the other.
• 2026-08-23: A further audit pass, and a worse class of error than the last: correcting the tables had left the summary, the FAQ, the structured data and the bundle READMEs carrying the old framing, which are exactly the surfaces that get quoted. Fixed here: four surviving “identical inputs” claims; “about a third of the notes” still in Limitations; the OLiMPiC median convention stated in one place and not the others; “recomputes the entire table” when the scorer cannot touch the OLiMPiC suite; a claim that staff and spelling accuracy were in the per-piece JSON when they were not (they are now); “Audiveris is behind on all 31” photographed pieces when it ties on one; and an earlier entry on this list whose historical figures a numeric sweep had silently rewritten, which is the opposite of what a changelog is for. Also newly disclosed: the photograph simulation does not resample, so it preserves full page resolution.
• 2026-08-23: Benchmark data published: ground truth, every system’s raw transcriptions, the scorer and the photograph degradation, so the tables can be recomputed. Recomputing Audiveris on OLiMPiC from those published files gave 64.4% median with usable output on 50 of 60, against the 65.4% median on 49 of 60 first published here; the figures on this page are now the ones the published files produce. Audiveris’s mean on that suite, 57.6%, is unchanged.
• 2026-08-23: Sheetly re-run on the same pinned inputs after fourteen pipeline changes: clean 90.8% → 97.03%, simulated photographs 83.3% → 96.00%, OLiMPiC 81.6% → 90.3% mean and 98.4% → 99.2% median, with usable output rising from 58 to 60 of 60. Audiveris 5.11.0 and oemer 0.1.x were not re-run and their July figures stand. Scored multi-page; the page-one figures that superseded these are in the entry above.
• 2026-08-22: First publication of the July 2026 run: Sheetly 90.8% clean, 83.3% photographed, 81.6% mean / 98.4% median on OLiMPiC with 58 of 60 usable; Audiveris 5.11.0 87.92%, 38.55%, 57.6% / 65.4% with 49 of 60; oemer 0.1.x 37.72% and 7.70% on the 31-piece suite. These are the figures as published that day, scored multi-page, and are left as they were rather than restated under the current convention.

Future runs will be added here with their dates rather than replacing these figures, so that the page stays a record and not a claim.

Ask Sheetly: a coach that has read your score

Accuracy is what makes the next part possible. Once the page has been read correctly, the coach in Sheetly holds the actual notes of it, so you can ask about your bar 12 and get an answer about what is written there.

bar 12 keeps falling apart, what am I doing wrong?

Bar 12 is where the left hand crosses over. You are almost certainly re-taking the thumb. Try 2–1–3 and keep the wrist moving right through the bar rather than resetting on the downbeat.

what should I drill tonight?

Bars 12–15, hands separately, at 60. That is the only place the texture changes; the rest of the page is a variation on the opening you already have.

We know of no other sheet-music scanner that does this. Every other system on this page hands you a transcription and stops.

Free to download · iPhone and iPad

Questions

Read next