Intelligent Document Processing: How It Works

Intelligent document processing earns its place when it removes retyping while keeping bad data out of your systems. Organize it around 4 jobs: classify the document, extract its fields, validate the values, and route exceptions to a person before filing or posting anything.

For an operations or finance lead, the first useful outcome is a checked draft record with its source attached. Automatic payment, contract interpretation and supplier approval are separate decisions. Start with the work that consumes your day: reading attachments, copying fields and finding the right folder.

Capabilities, licences and hosted API prices verified against the makers' documentation on 29 October 2026. The pilot below is a proposed evaluation method, not a product benchmark.

How Intelligent Document Processing Works

IDP is a workflow that turns documents into usable records and sends uncertain cases for review. Think of a records clerk with a checklist: identify what arrived, copy the relevant information, compare it with the rules, and give the unresolved folder to someone who can decide.

The input might be a structured form with fixed boxes, a semi-structured invoice whose layout varies by supplier, or an unstructured email. The destination might be your accounting system, a contract register or a case-management queue. AWS's IDP overview describes classification, extraction, validation and integration as parts of this broader process.

A useful invoice workflow looks like this:

1. Classify. Decide whether the attachment is an invoice, credit note, statement or something else. Separate a combined file into its component documents before extracting fields.
2. Extract. Read the supplier, invoice number, dates, currency, totals and line items. Preserve the original file and the page or region supporting each value.
3. Validate. Check required fields, arithmetic, duplicates and the supplier record. A readable total can still belong to the wrong invoice.
4. Route. File an accepted record in the permitted destination. Send a missing purchase order, inconsistent total or uncertain classification to a named reviewer.

These are responsibilities to design, not features to assume every extraction API delivers. An API, the connection software uses to request a service, can return fields without knowing your approval policy.

Keep the file identifier attached to the record throughout. Otherwise, a correct extraction can become an operational failure when someone cannot find the document that justified it.

What LLMs and Vision Models Changed About OCR

The change is that extraction can use meaning and page context instead of relying only on where a field appears. Optical character recognition, or OCR, reads characters from images. A template-based extractor then looks in known positions or follows layout rules. A large language model, or LLM, interprets language; a vision-language model also takes images as input.

For example, a supplier might move “Amount due” from the lower right to a summary box near the top. A fixed-position rule needs adjustment. A model instructed to find the payable total can use the label and surrounding context, although it can still choose the wrong amount.

Google Document AI's Custom Extractor supports foundation-model extraction from a defined schema, the named fields you want returned, with zero-shot and few-shot options. Those mean using no task examples or a small set of examples. Managed extraction has therefore already moved beyond the old template-only model. Google's Custom Extractor documentation explains the available methods.

Structured outputs address a different problem: keeping the response in the required data shape. A string in a date field or a numeric total is easier for software to process, but its format does not establish that it came from the right place. Keep schema validation, which checks structure, separate from business validation, which checks whether a value is acceptable.

Choosing the OCR Model

Start with a hosted extractor for routine invoices and forms; evaluate open models when self-hosting or cross-page continuity is the requirement. The decision is what your next step needs: readable text, a reconstructed table, or named business fields. These recommendations reflect documented capabilities, not a head-to-head accuracy test.

Since October 27, 2026, Baidu's Unlimited OCR 2 release has added an open candidate for long-document parsing alongside MarathonDocBench, its evaluation dataset. The model card lists MIT. The benchmark's own licence section instead specifies Apache-2.0 for annotations and tooling references, with page images provided for academic and research use. Do not assume the model's licence grants reuse rights over the benchmark's source documents.

Scans: Choose Text or Structured Evidence

Mistral OCR is a useful hosted starting point when you want document text and layout for a downstream model. Its OCR processor accepts PDFs and images and returns structured content; OCR 4.1 adds paragraph bounding boxes, structural labels and block-level confidence. Those coordinates can help a reviewer find the source of an extracted value. They do not verify that your application picked the right value.

For scans inside an existing cloud workflow, Azure Document Intelligence Read extracts printed and handwritten text. Google Document AI Enterprise Document OCR also offers image-quality assessment, useful when you need a recapture queue. Use OCR text as an intermediate result; choose an invoice extractor when the required output is a supplier, payable total and line items.

Tables and Forms: Match the Output to the Record

Begin with Azure's invoice model or Google's Invoice Parser for invoice fields, and evaluate layout extraction when the table itself is the deliverable. Azure Layout extracts text, tables and selection marks. Google Form Parser extracts key-value pairs, checkboxes and tables; Google's Invoice Parser returns header and line-item fields.

Mistral's OCR processor can return tables as Markdown or HTML. That is useful when a contract schedule needs to remain a table before another step extracts obligations. For every candidate, check merged cells, repeated headers and row alignment against the original. A well-formatted table can still attach an amount to the wrong item.

Handwriting: Test the Language and the Correction

Shortlist Azure Read and Google Enterprise Document OCR for handwritten forms, then test the actual handwriting and language you receive. Their documentation describes handwriting support; Mistral also advertises handwriting extraction. This is a capability statement, not evidence that any one wins on your forms. The open-model cards do not establish a comparable handwriting winner either. Include overwritten totals, cursive names and crossed-out entries in the same evaluation for Unlimited OCR 2 and DeepSeek-OCR if you need an open deployment.

Long Documents: Test Continuity Across Page Breaks

Unlimited OCR 2 belongs on the shortlist when a contract or invoice schedule continues across pages. Baidu's model card documents ordered multi-page input in one request, so parsing can use cross-page context. It also names a deployment tradeoff: smaller page windows lower peak memory but can hurt merging where continuing blocks cross a window boundary. Baidu supplies post-processing code that converts layout blocks into Markdown and merges cross-page table fragments.

MarathonDocBench provides a harder test than isolated pages: its card describes 100 documents spanning 100 to 300 pages. Baidu reports 77.06 Overall for Unlimited OCR Works 2 on the benchmark's own leaderboard. That is a document-parsing benchmark score, not an invoice-field accuracy percentage. Its baseline method sends whole documents in one request and counts failures as empty predictions, so it does not establish which production pipeline will work best on your files.

For hosted APIs and DeepSeek-OCR, apply the same continuity test: preserve page order, check the last row before a break against the first row afterward, and verify that no clause or subtotal disappeared. Accepting a long file and correctly reconstructing its relationships are separate requirements.

Self-Hosting: Pick the Model You Can Operate

DeepSeek-OCR is a useful local baseline for image-to-Markdown work; Unlimited OCR 2 is the more targeted candidate for cross-page parsing. Here DeepSeek-OCR means the original deepseek-ai/DeepSeek-OCR checkpoint. Its model card documents Transformers and vLLM inference, its repository includes PDF processing, and its weights repository licence is MIT. Baidu documents Transformers for testing and a customized SGLang build for serving Unlimited OCR 2. Budget operational effort for either: deployment, monitoring, version control and output validation remain yours.

For hosted routes, price the endpoint your job requires. Mistral OCR 4.1 lists $4 per 1,000 pages for standard OCR in its API pricing. Azure S0 Read lists $1.50 per 1,000 pages for the first million pages; Layout and prebuilt invoice extraction list $10 per 1,000 pages, subject to region and agreement, on Azure's pricing page. Google Invoice Parser lists $0.10 for up to 10 pages in a document, with further blocks billed separately, on Google's pricing page. Plain OCR and invoice-field extraction are different purchases; compare the work left after each response, not only the page rate.

Keep your current extractor if it already meets the acceptance rules. Add an open-model trial when control over hosting or measured failures on long documents justify the work, then use the pilot below to decide whether to switch.

Intelligent Document Processing Use Cases

Choose the first workflow by how clearly you can define a correct answer. A bounded extraction task is easier to evaluate than an instruction to “handle all our paperwork.”

For accounts payable, produce draft bill records from invoices, match them to suppliers and flag duplicates. Keep a bank-detail change outside automatic acceptance even when the printed account number is perfectly readable.

For contracts, extract parties, stated dates and the passage describing renewal. Distinguish copying a notice period from deciding how it applies across amendments. An operations lead can use the first to prepare a register; the second needs an accountable reviewer.

For forms, capture typed entries and selection marks, then check completeness. An onboarding form missing a required signature should remain incomplete rather than acquire an inferred answer.

For emails, identify the request and link it to its attachments. An email saying “please use the revised invoice” requires the workflow to preserve that relationship, not merely extract both attachments into duplicate records.

The same pattern can support claims intake, shipment paperwork or employee records. The test stays concrete: what value is required, where is its evidence, and who resolves disagreement?

Intelligent Document Processing Software: Build Versus Buy

Buy the finished workflow when the job is standard; build around extraction services when your rules and destinations are the unusual part. The distinction is how much intake, validation, review and integration your organization must own.

A managed service is a component, not automatically a ready-to-run finance workflow. Conversely, “custom” does not mean training a model from scratch. You can build your own intake and review process around a hosted extractor.

For a buyer, the decisive demonstration is your reviewer completing an exception with the original document beside it. A successful upload animation says little about the work left afterward.

Intelligent Document Processing Vendors and the LLM-Native Route

Compare these routes on your document mix, evidence requirements and review workload. The descriptions below reflect the makers' documented capabilities; the fit judgments are recommendations, not an accuracy ranking.

Google Document AI: Processor-Based Extraction With Generative Options

Google Document AI is Google Cloud's document-processing service. Its processor catalog includes OCR, Invoice Parser, Custom Extractor, Custom Classifier and Custom Splitter. Invoice Parser extracts header and line-item fields; classification and splitting handle document identity and boundaries. Google's processor catalog describes the separate roles.

Start with the matching prebuilt processor, then evaluate Custom Extractor if your required fields or document types do not fit. Its generative route supports schema-driven extraction and optional examples or fine-tuning. Google also provides exact and fuzzy matching in evaluation, so agree on acceptable normalization before comparing results. Custom Extractor documentation.

The wall is workflow ownership: returning an invoice amount does not match a supplier, enforce your policy or close an exception. Confirm the chosen processor version's language, region and document support before the pilot.

Azure AI Document Intelligence: Prebuilt and Custom Models

Azure AI Document Intelligence is Microsoft's document extraction service. Its documented options include Read and Layout, prebuilt invoice and receipt models, custom template and neural extraction, and custom classification. Microsoft describes custom templates for static layouts and neural models for more varied documents. Microsoft's model overview.

For an Azure-based operations system, begin with the prebuilt model that fits the document. Add custom classification when incoming packets contain different document classes, and assess custom extraction when the required fields differ.

The wall is assuming layout recognition completes the business record. Layout can identify text, tables and selection marks; your application still has to reconcile outputs with its destination. Microsoft explicitly discusses post-processing page-spanning tables into a single table in its Layout guidance.

AWS Intelligent Document Processing: Amazon Textract

Amazon Textract is Amazon Web Services' extraction service for document text and data. AWS documents text detection, form and table extraction, and specialized invoice and receipt processing. Its expense analysis can recover fields such as invoice date, number and total without supplier-specific templates. Textract's invoice and receipt documentation.

Textract is a reasonable first candidate when the surrounding application already runs on AWS and the document fields match its operations. Select the relevant operation instead of treating every document as plain text.

The wall is assuming that a confidence score settles a business decision. AWS recommends setting thresholds according to the sensitivity of the application and sending uncertain results for scrutiny. It also warns about inconsistent extraction from merged cells and irregular tables. Textract best practices.

The LLM-Native Route: A Model, a Schema and a Review Queue

OpenAI API is one documented example of the LLM-native approach: software sends a document to a model and requests fields in a defined structure. For PDFs, its vision-capable input path supplies both extracted text and page images. File-input documentation.

Your application defines the fields, requires evidence references and decides what happens when a value is absent. Use a missing-value state with a reason such as “not present” or “illegible”; do not force the model to fill every blank. Check evidence references against the original file rather than assuming a generated page citation is correct.

OpenAI documents schema adherence for supported structured outputs, while explicitly noting that values can still contain mistakes and that refusals or incomplete responses need handling. Structured Outputs documentation. Our structured outputs guide explains the contract between the model and your application.

Choose this route when contextual extraction is the missing capability and someone can maintain the workflow. The wall is operational: a prompt and JSON response leave you responsible for evaluation, retries, review, storage and safe destination writes.

Dext: Keep It in the Bookkeeping Lane

Dext is a bookkeeping document-capture application, a relevant option when the backlog is receipts and bills. Its documentation describes extracting supplier or customer, dates, totals, tax and invoice numbers into an item detail view for review, editing and export to connected accounting software. Dext's extraction guide.

For a finance lead whose problem ends at a checked bookkeeping record, that is a more relevant starting point than commissioning a general document system. This recommendation does not extend to interpreting contracts or orchestrating arbitrary forms and emails. Use the AI accounting software comparison when the decision is about the bookkeeping application itself.

Run a Pilot That Measures Fields Before Trust

Score the values the business needs before allowing the workflow to act on them. A document that looks correctly transcribed can still contain the one mistake that matters: the wrong currency, supplier or payable amount.

1. 1

   Define one document class and its required fields

   Start with a bounded job such as supplier invoices becoming draft bills. Name every required field, its source and the action it can authorize. Keep extraction separate from derived values: a stated due date and a due date calculated from payment terms are different results.

2. 2

   Create human-checked answers and hold out a test set

   Collect representative historical documents, including unfamiliar suppliers, poor scans, credit notes and difficult tables. Ground truth means the correct answers established by people. Have ambiguous or critical labels checked by another reviewer. Keep the evaluation documents separate from examples used to tune prompts or train the extractor, including near-duplicate documents from the same transaction.

3. 3

   Agree on matching rules before looking at results

   Normalize harmless differences such as surrounding spaces or a known date format. Do not use loose text similarity for money, account identifiers or invoice numbers. Preserve currency and decimal meaning. For line items, score both the values and their assignment to the correct row; a correct amount attached to the wrong item is wrong.

4. 4

   Run candidates on the same untouched documents

   Save each raw response before manual correction, the model or processor version, prompt/schema version, processing failure and source reference. Count unreadable files, timeouts and empty extractions in the operational results. Do not quietly remove them from the denominator.

5. 5

   Route exceptions and operate in shadow mode

   Have the proposed workflow produce drafts alongside the existing process. Measure the corrections and review time. Only enable automatic acceptance for a document subset whose held-out results meet thresholds the process owner set in advance; keep high-impact actions behind their own approval.

Measure Fields, Complete Records and Silent Errors Separately

A single average hides the location and consequence of errors. Use a scorecard with separate results for each field and for each difficult document group.

Report absent optional fields separately so that hundreds of correctly empty boxes cannot inflate performance. If you also report recall, define it as correctly recovered required values divided by all required values. Evaluate document classification and packet splitting separately; a perfect extraction from the wrong document is still a failed record.

Illustrative arithmetic, not measured product performance: suppose 200 invoices each require 8 fields, giving 1,600 expected values. If 1,590 values are correct but the remaining 10 are payable totals on different invoices, only 190 invoices are fully correct. A favorable aggregate field score cannot authorize all the bills.

Measure silent errors by auditing records the workflow would have accepted, not only by inspecting its review queue. Start by checking every pilot output; later sampling must remain representative of what is being accepted. If no records were auto-accepted, the silent-error rate is undefined, not a perfect result. Apply the same rule to precision when no values were populated.

Benefits of Intelligent Document Processing: Count Work Removed

The operational benefit is fewer minutes spent obtaining a correct record. Compare manual handling with the full proposed process: intake, model processing, correction, review, export and rework. Also track turnaround time and the age of the oldest unresolved exception. Fast extraction with a growing queue has moved the backlog rather than cleared it.

Give reviewers the source page, highlighted evidence where available, extracted value, failed rule and destination. Record the correction and reason. The human-in-the-loop AI guide covers queue ownership and escalation; assigning someone to resolve exceptions is part of launching the workflow.

Where IDP Breaks: Handwriting, Tables and Scans

Put difficult inputs into the pilot deliberately. An evaluation made only of clean digital invoices tells you little about a phone photo, an annotated form or a table that continues onto another page.

Handwriting needs its own test group. Crossed-out amounts, cursive notes and overlapping stamps create ambiguity even when a product supports handwriting. Support can also vary by language: Amazon Textract documents handwritten character recognition in English only. Textract's limits. Route an uncertain handwritten correction to review rather than allowing it to silently replace a printed amount.

Multi-page tables need continuity checks. A page break can separate a row, repeat a header or introduce a subtotal that resembles a line item. Check row counts, continued descriptions and totals across the whole document. Microsoft's Layout guidance explicitly discusses post-processing pages into one table. That is a reason to test the chosen model and stitching logic, not a claim that every model fails on every multi-page table.

Low-quality scans need a recapture path. Blur, cropped edges and compression can remove information that a model cannot reliably recover. AWS recommends high-quality images, ideally at least 150 DPI, and advises against downsampling already supported files. Textract best practices. Preserve the original; ask for a clearer copy when the evidence is unreadable.

Also test missing pages and combined packets. A record can pass every field-format check while being incomplete. Keep an exception reason for missing evidence, not just a generic processing failure.

What Is Overhyped

“Reads documents” is not the same as “can safely complete the transaction.” Three assumptions cause avoidable mistakes.

First, structured output is not verified output. Constraining a response to a schema cannot establish that the extracted bank account belongs to the approved supplier.

Second, confidence is not a universal acceptance policy. A service score, a model's self-reported certainty and measured correctness on your own documents are different things. Calibrate the acceptance rule against held-out results and error consequences; do not treat a model-generated confidence number as a measured probability.

Third, saving corrections does not necessarily retrain the model. Use corrections to diagnose failures and improve rules, prompts or training data, then re-evaluate the change before rollout. Keep a regression set, a fixed collection of previously checked documents, so an improvement for one supplier does not hide a failure for another.

Who Should Act Now, Wait or Keep Their Current Setup?

Act now when repetitive intake has a clear destination, checkable fields and a person who owns exceptions. For an operations lead, the next move is to assemble a representative sample and name the required fields. For a finance lead, make the initial output draft records; keep payment authorization separate.

Builders should start with the simplest extractor that fits. A solo technical builder can compare a managed model with an LLM path on the same held-out sample. A mid-market technology lead should require evidence links, review ownership, retention rules and repeatable evaluation before connecting the workflow to production records. Keep document text as input data; it must not be allowed to rewrite the application's instructions or permissions.

Buyers should wait if nobody can define a correct record or own the review queue. A funded founder with that gap should assign the process owner before commissioning custom automation. A team whose existing template process handles stable forms reliably has no reason to replace it merely because vision models exist.

On Monday, choose one document class, collect the historical sample, write the acceptance rules with the reviewer, and run the proposed workflow into a draft destination. Expand only when the field results and remaining workload justify it.

Questions About IDP

What is intelligent document processing?

It is a workflow for classifying documents, extracting information, validating it and routing the resulting records or exceptions. The useful output is checked data with a traceable source.

What is the difference between OCR and intelligent document processing?

OCR reads text from an image. IDP includes the classification, field extraction, checks and routing needed to use that text in a business process. OCR can be one component inside it.

What is the best intelligent document processing software?

Choose by the job and the work left for reviewers. For receipts and bills, begin with a bookkeeping application such as Dext. For custom intake, evaluate a managed extractor or an LLM workflow against the same human-checked documents.

Is IdP considered AI?

Intelligent document processing commonly uses AI for classification and extraction, alongside ordinary business rules. In other contexts, IdP means identity provider, a different concept.

Who are the top IDP vendors?

Google Document AI, Azure AI Document Intelligence and Amazon Textract are managed routes covered here. This is a capability comparison, not a market-share or accuracy ranking; their suitability depends on the documents and workflow.

Which AI is best for document processing?

The best candidate is the one that meets your required-field and complete-record acceptance criteria with manageable review effort. A model that reads unusual layouts well may still need additional checks for tables and missing fields.

Is there a better AI than ChatGPT?

For repeatable document intake, compare an implemented workflow with your current process, rather than judging a chat answer alone. You need predictable fields, source evidence, exception handling and a controlled destination whichever model you choose.

How much does Document AI cost?

Google Document AI pricing varies by processor and volume; use the official pricing page for the processor you plan to pilot.

Does Google Docs use your writing for AI?

For Google Workspace, Google says customer data is not used for generative AI model training outside the customer's domain without permission. Check the Workspace AI privacy policy for your account and feature; Google Docs is separate from the Document AI service discussed here.

For practical guides to putting AI into business workflows, join the newsletter.