AI / LLM Integration

Embedding large language models into production automation workflows — not as chatbots, but as transformation engines.

There is a wide gap between “using ChatGPT” and “integrating LLMs into production automation pipelines.” The first is a conversation. The second is an engineering discipline — involving prompt design, structured output contracts, confidence handling, agentic orchestration, retrieval systems, and rigorous testing against real business data.

I crossed that gap in production at ArcelorMittal Luxembourg in 2024, and have been building on those foundations ever since — adding RAG architectures, tool-calling agents, and LangGraph-based workflows to my stack. My model coverage now extends across OpenAI, Anthropic, Mistral, and locally-hosted open-source models via Ollama. This post covers every layer of how I work with LLMs in automation contexts.

The most important design decision when integrating an LLM is treating it as one step in a larger pipeline — not as the whole product. In every system I have built, the LLM sits between structured data retrieval and structured output validation. It never owns the I/O contract. That contract belongs to the surrounding automation layer.

Prompt engineering in a production context is not about clever phrasing — it is about designing deterministic, testable input contracts that produce consistent structured output. I work with system prompts that define output schema explicitly, few-shot examples drawn from real production data, and chain-of-thought instructions for multi-step reasoning tasks.

Below is an example prompt used to extract steel products from client emails — illustrating the structured output approach with Azure OpenAI:

from openai import AzureOpenAI
import json
import re

client = AzureOpenAI(
    azure_endpoint=AZURE_ENDPOINT,
    api_key=AZURE_API_KEY,
    api_version="2024-02-01",
)

SYSTEM_PROMPT = """
You are a structured data extraction engine for steel products.
Extract every steel product and its quantity from the email.
Rules:
- Return ONLY valid JSON
- Format: {"product_name": quantity_or_null}
- Quantities are integers (tonnes)
- If no quantity → null
- No explanations, no text outside JSON
"""

def extract_products(email_body: str) -> dict:
    response = client.chat.completions.create(
        model="YOUR_DEPLOYMENT_NAME",  # Azure: deployment name, not model name
        temperature=0,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user",   "content": email_body},
        ],
    )
    raw = response.choices[0].message.content
    # safer JSON extraction
    match = re.search(r"\{.*\}", raw, re.DOTALL)
    if not match:
        raise ValueError(f"Invalid JSON response: {raw}")
    return json.loads(match.group())

Beyond simple inference, I build agentic workflows where the LLM decides which tools to call and in what order — dynamically routing through a pipeline based on the content of the input. This is qualitatively different from a fixed pipeline: the model reasons about the task and selects the appropriate execution path.

LLMs hallucinate when asked about data they were not trained on — internal product catalogues, historical order data, company-specific documents. RAG solves this by retrieving relevant context at inference time and injecting it into the prompt. The model reasons over real, current data rather than its training weights.

At ING, I built and maintained a risk assessment classifier used for Customer Due Diligence (CDD) — estimating organisational risk across financial security, fraud, misappropriation, and harmful entity detection. This was a traditional ML pipeline (Scikit-Learn) but required the same discipline as LLM work: careful feature engineering, iterative retraining as business rules evolved, and rigorous validation against regulatory requirements.

A critical real-world constraint emerged: the model’s statistical decisions could not be taken directly to production without business approval. Patterns that the classifier learned implicitly had to be formalised into explicit, deterministic rule sets — human-readable logic that the business could review, challenge, and sign off on. In practice, this meant working back from the model’s behaviour to codify its reasoning as manual rules, which were then implemented alongside the ML layer. This experience sharpened my understanding of the gap between model accuracy and operational trustworthiness — a lesson that directly shapes how I design LLM output validation and human-review lanes today.

This background in ML classification directly informs how I design LLM-based classification tasks today — treating prompt + model as a classifier, measuring precision/recall, and building human-review lanes for low-confidence outputs.

Job hunting at scale has the same problem as enterprise search: most of the corpus is irrelevant noise. Sending every posting through a full LLM prompt costs tokens and time. The insight behind this project is to apply a cheap embedding pre-filter first — exactly the same pattern used in RAG retrieval — and only engage the model on the small fraction of offers that pass the similarity threshold.

The pre-filter is intentionally model-agnostic and runs in milliseconds per offer — no API round-trip, no prompt, just vector arithmetic. This makes it practical to run against hundreds of scraped postings without meaningful cost. The Streamlit output layer shows each offer ranked by similarity score alongside a short gap summary, so the human reviewer immediately sees why an offer scored high or low, not just that it did.

The embedding pre-filter is the foundation. The next layer will pass shortlisted offers and the current CV into an LLM prompt that identifies specific gaps (missing keywords, under-represented skills, tone mismatches) and generates targeted, per-offer suggestions for CV sections to rewrite. The output will be a ranked list of offers, each paired with a concrete, actionable edit brief — not generic advice, but diff-style suggestions tied to the specific language of that posting.