Turning unstructured documents into structured, machine-readable data — at enterprise scale.
Most enterprise data does not live in neat database tables. It lives in scanned PDFs, faxed contracts, handwritten forms, identity documents, and chemical research reports — formats that no SQL query can touch. Document Intelligence is the engineering discipline of bridging that gap: extracting, validating, and routing information from unstructured sources into systems that can actually use it.
I have built and maintained production document intelligence pipelines across two employers, four Western European markets, and multiple document types. This post covers the full stack — tools, architectures, accuracy patterns, and real project outcomes.
The core challenge
Document intelligence is deceptively difficult. A scanned PDF of a Dutch power of attorney document looks nothing like a Belgian one — different layout, different fonts, different field positions, different languages, and potentially a handwritten signature that must be detected as present or absent. Any pipeline that handles one handles none of the others unless explicitly engineered to do so.
The core pipeline pattern is always the same, but every layer requires deliberate engineering decisions:
OCR engines I have used in production
| Engine | Best for | Layout variance | Deployment |
|---|---|---|---|
| ABBYY FlexiCapture | High-volume structured forms | Low tolerance | On-premise / server |
| ABBYY Vantage | Semi-structured, variable layouts | Medium tolerance | Cloud / hybrid |
| Azure Document Intelligence | Azure-integrated pipelines, APIs | High tolerance | Azure cloud |
| Azure Computer Vision OCR | Raw text extraction, preprocessing | Very high | Azure cloud |
Document classification
Before any field can be extracted, the pipeline must know what type of document it is looking at. At ING, I built a classifier that distinguished between power of attorney documents and identity documents across four markets — each with different layouts and languages. The classifier routed documents to the correct extraction model before any OCR ran.
Field extraction patterns
Extraction strategy depends entirely on document structure. I use three patterns depending on the document type:
from azure.ai.formrecognizer import DocumentAnalysisClient
from azure.core.credentials import AzureKeyCredential
client = DocumentAnalysisClient(
endpoint=“https://your-resource.cognitiveservices.azure.com”,
credential=AzureKeyCredential(api_key)
)
poller = client.begin_analyze_document(“prebuilt-document”, document)
result = poller.result()
for kv_pair in result.key_value_pairs:
if kv_pair.key and kv_pair.value:
fields[kv_pair.key.content] = kv_pair.value.content
Signature detection
One of the more unusual requirements at ING was detecting the presence of a handwritten signature on power of attorney documents. A document without a signature is legally invalid and must be rejected before it enters the processing queue. I built a computer vision step using OpenCV and Pillow that analysed pixel density in defined signature zones — a simple but effective binary classifier that ran ahead of the full OCR pipeline.
Validation & post-processing
Raw OCR output is never trusted directly. Every extracted field goes through a validation layer — format checks (dates, ID numbers, postcodes), cross-field consistency checks, and confidence score thresholds. Low-confidence extractions are flagged for human review rather than silently passed downstream.
Production deployments
1
Core extraction goal: detect all instances of account holders and proxies (representatives granted authority) within a single document, including cases with multiple parties. For document types that required it, a pre-processing gate detected the presence of a handwritten signature using pixel density analysis (OpenCV + Pillow) — documents without a valid signature were rejected before entering the OCR queue. Achieved 70% end-to-end accuracy across all markets.
2
Each country required its own extraction model due to differences in document layout, field encoding, and script. The pipeline covered a broad geographic scope well beyond the Western European core, handling document variants from Central and Eastern Europe and other regions.
My role was to design and maintain predefined extraction rules for each report type as new variants appeared, mapping extracted fields into standardised columns in a central database. This required continuous rule authoring, regression testing when layouts shifted, and ensuring that data from heterogeneous sources landed correctly in a unified schema for downstream reporting and analytics.
Extended tech stack
Python image processing & OCR libraries
All image pre-processing in my pipelines is written in Python. Pillow handles format conversion, rotation correction, and colour normalisation before OCR engines receive the image. The four most widely used open-source Python OCR libraries are:
