Nevertheless, these promotional claims do not all the time match real-world efficiency, in keeping with latest checks. “I am usually a fairly large fan of the Mistral fashions, however the brand new OCR-specific one they launched final week actually carried out poorly,” Willis famous.
“A colleague despatched this PDF and requested if I may assist him parse the desk it contained,” says Willis. “It is an previous doc with a desk that has some complicated structure parts. The brand new [Mistral] OCR-specific mannequin actually performed poorly, repeating the names of cities and botching numerous the numbers.”
AI app developer Alexander Doria additionally not too long ago identified on X a flaw with Mistral OCR’s potential to grasp handwriting, writing, “Sadly Mistral-OCR has nonetheless the same old VLM curse: with difficult manuscripts, it hallucinates fully.”
In accordance with Willis, Google at the moment leads the sphere in AI fashions that may learn paperwork: “Proper now, for me the clear chief is Google’s Gemini 2.0 Flash Professional Experimental. It dealt with the PDF that Mistral didn’t with a tiny variety of errors, and I’ve run a number of messy PDFs by it with success, together with these with handwritten content material.”
Gemini’s efficiency stems largely from its potential to course of expansive paperwork (in a kind of short-term reminiscence known as a “context window”), which Willis particularly notes as a key benefit: “The scale of its context window additionally helps, since I can add massive paperwork and work by them in components.” This functionality, mixed with extra strong dealing with of handwritten content material, apparently offers Google’s mannequin a sensible edge over opponents in real-world document-processing duties for now.
The drawbacks of LLM-based OCR
Regardless of their promise, LLMs introduce a number of new issues to doc processing. Amongst them, they’ll introduce confabulations or hallucinations (plausible-sounding however incorrect data), by accident observe directions within the textual content (pondering they’re a part of a person immediate), or simply usually misread the information.
