A finance lead uploads a scanned invoice to your product. The extraction returns every field correctly spelled, in valid JSON, matching your schema exactly. The line-item table comes back as one long run-on string with the columns interleaved. Nobody notices for five weeks, because the text was right.
That failure decides whether AI document processing works inside your SaaS, and it has almost nothing to do with which model you picked. Document extraction is three decisions stacked on each other: how the PDF becomes text, which model turns that text into structured fields, and where that model runs. Most teams argue about the third one first, usually phrased as "OpenAI or local," then wonder why accuracy sits stuck at eighty-something percent for a month.
I want to take those in the order that actually moves the number.
Text Accuracy and Structure Accuracy Are Different Numbers
A parser can read 75% of the words on a page correctly and still destroy 87% of that document's structure. Those are two independent scores, and demos only ever show you one.
Applied AI's December 2025 PDF parsing benchmark pushed 800+ documents across six domains through 17 parsers: seven frontier LLMs, four commercial APIs, and a spread of open-source tooling. The pair of numbers that matters is GPT-4o-mini's. It reached 75% edit similarity on text and 13% tree similarity on structure. Gemini 3 Pro and GPT-5.1 both landed at 42% tree similarity, LlamaParse at 39%. Read that again. A model can be three-quarters right about the words while being almost entirely wrong about which cell they belonged to.
Downstream, that is the whole game. If extraction feeds a RAG index or an accounting ledger, the table structure carries the meaning. A row that loses its column alignment is not partially correct. It is a confidently formatted lie.
Dedicated parsers win here because they are solving a different problem. Docling ships a TableFormer model that recovers row and column structure and merged cells as an explicit task with its own training objective. A general vision-language model is doing next-token prediction over a rendered page and hoping. No prompt closes that gap.
Parse with something that understands layout, then hand clean markdown to a language model for the semantic part. That ordering does more for extraction accuracy than any model swap will, and it sits entirely outside the OpenAI-versus-local argument everyone starts with.
Does a Local Model Actually Match GPT-4o on Extraction?
A local model matches a frontier model closely on standardized fields like dates and totals, and falls meaningfully behind on messy narrative ones that require interpretation.
That comparison is usually made badly, so it is worth reading one that was made well. A 2026 Applied Sciences study held everything constant except the model: Docling for PDF-to-markdown, a two-pass anonymization step, identical prompts, identical schema constraints, near-zero-temperature decoding. Then it swapped in GPT-4o, GPT-OSS-120B, and Llama-3.1-8B-Instruct and measured completeness and content similarity across 2,280 multilingual CVs. GPT-4o held full schema coverage as the baseline. The open-weight models reached 73–79% completeness and 59–72% content similarity depending on how complex the section was.
The useful result is not the size of the gap. It is where the gap lives. Llama-3.1-8B performed strongly on standardized sections like contact details and legal fields, anything with a predictable shape. GPT-OSS-120B did better on infrequent narrative fields, the parts where a human reader would have to interpret rather than copy.
That maps onto a rule you can act on this week. Fields that are structurally predictable (dates, totals, VAT numbers, addresses, policy numbers) are fine on a small local model. Fields that require reading intent out of prose still earn a frontier model its per-token price. Most document workloads are roughly 80% the first kind.
So the question was never "which model." It is which fields go to which model. Routing beats picking.
What Does Self-Hosting a Document Model Really Cost?
Budget three to five times the GPU sticker price, because that sticker assumes a utilization rate real document traffic will never hit. Rented compute costs the same idle as it does at full load.
Particula's May 2026 break-even analysis puts sustained real-world GPU utilization at 40–65%. Apply that to an H100 carrying a $2.69/hour spot sticker and the effective cost lands at $4–6/hour. Their conclusion is blunt: below roughly $15–20K/month of premium API spend with a single team, self-hosting loses on every spreadsheet, because the headcount line alone exceeds the entire premium bill. Realistic break-even for pure self-hosting sits at $50–80K/month, not the $20K the naive math produces.
Document processing has a specific problem with this. Its traffic is spiky in the worst possible shape: nothing for six days, then a client firm uploads four hundred contracts on a Monday morning. Then silence again. That pattern leaves a reserved GPU idle most of the month while you pay for every hour of it.
| Workload shape | Cheapest correct answer | Why |
|---|---|---|
| Under 50k pages/month, mixed fields | Hosted API on a cheap tier | Per-page cost never approaches one engineer-week |
| Spiky batch, tens of thousands of pages | Serverless GPU (Modal, Replicate) | You pay for the burst, not the calendar |
| Sustained high volume, uniform fields | Reserved GPU plus an open-weight model | Utilization is finally high enough to matter |
| Any volume, data cannot leave jurisdiction | Local or in-region VPC, whatever it costs | Compliance is not a line in the cost model |
The last row is the only one where cost stops being the deciding input.
The July Price Cut Moved the Line
On 30 July 2026 OpenAI cut GPT-5.6 Luna from $1.00/$6.00 to $0.20/$1.20 per million tokens, an 80% drop, and trimmed Terra by 20%. CloudZero's pricing tracker has the before-and-after. Every break-even spreadsheet built before August is now wrong in the same direction. Re-run yours before you sign a GPU commitment.
So When Does Local Actually Win?
Local wins when the data cannot legally leave your jurisdiction, or when sustained volume makes a per-page cost gap compound across millions of pages. Only the first survives a price cut.
Cost arguments for self-hosting carry an expiry date, and July just demonstrated it. Compliance arguments do not expire. If you are processing UK or EU health records, or legal discovery material where a data processing agreement names specific sub-processors, the deciding question is not what inference costs per page. It is whether the architecture is defensible when a customer's counsel asks who sees the file.
Here is how that shaped a real build. On Callidus, the multi-tenant clinic SaaS I built for UK aesthetic clinics between February and April 2026, the document surface was consent forms and patient records. Clinical data, UK GDPR, six roles with different access to clinical and financial records, every read path routed through shared role helpers rather than inline checks. Adding document understanding to a product shaped like that is not a model-selection exercise. It is a question about which processor sees a patient's treatment history, and the answer has to hold up in a contract rather than a benchmark table.
Actually — that understates the engineering side. The eval still matters enormously. It just runs second, on a shortlist compliance has already filtered down.
Build the Eval Rig Before You Choose Anything
You cannot pick a model for a document workload you have not measured, and building the measurement takes about a day.
- Collect 50 real documents from production, including the ugly ones. Scanned at an angle, photographed on a phone under bad light, three generations of the same template. If your sample is all clean digital PDFs, the eval will recommend the wrong parser with total confidence.
- Label the ground truth by hand. Fifty documents, the exact JSON you want out of each one. This is the tedious part and it is the entire reason the rig has value, because nobody else knows what your fields are supposed to mean.
- Score structure separately from text. Field-level exact match for scalars, something structure-aware for tables. The benchmark above exists precisely because those two numbers diverge, and one blended accuracy score will hide the divergence from you until a customer finds it.
- Run the cheapest viable configuration first. A layout parser plus a small model. That is your floor, and it is frequently good enough for the standardized 80% of your fields.
- Route, do not upgrade. When a field fails, send that field to a stronger model instead of moving the whole document up a tier. Per-page cost across the benchmark spanned $0.001 to $0.058, so a blanket upgrade costs something like 58 times what a targeted one does.
- Re-run the rig monthly against the same 50 documents. Prices move, models get deprecated, and your document mix drifts every time you sign a customer in a new vertical.
If you are choosing infrastructure from scratch rather than retrofitting it, my opinionated SaaS MVP stack covers what pairs cleanly with what. And the broader question of which AI features are worth adding to a business product is worth settling before you build any of this, because document extraction is rarely the feature users ask for first.
The Line Item Nobody Budgets For
Extraction confidence is a product decision, not a model one. Every pipeline gets fields wrong, and what separates a feature people trust from one they quietly stop using is whether the interface tells them which fields to check.
Surface a per-field confidence score and let a human correct it in two clicks. Store what they changed. That correction log becomes the highest-value eval and fine-tuning data you will ever own, and it costs nothing to collect, because your users are already doing the work of finding the errors. Most teams throw it in a log file and never look at it again.
Start with the fifty documents. Not the model comparison, not the GPU quote — pull fifty real files out of production this week, including the ones that arrived as phone photos, and label by hand what you want out of them. Then ask the question that decides everything downstream: how many of those fields were predictable enough that a small local model would have gotten them right, and how many genuinely needed something expensive?
If you are earlier than that and still working out how a subscription product is structured, start with what the SaaS delivery model actually involves. And if your question is less about what you ship than about how you build it, my AI-augmented development workflow covers the day-to-day version.
