LLM features wired into real products and real data

AI integration

Contract AI engineering for production products: retrieval over your documents, structured extraction, assistants and agent workflows, with the evaluation suite, guardrails and cost controls needed to run them for real users.

2 min read

Key takeaways

  • The engineering difficulty in AI features is rarely the model call. It is evaluation, failure handling, cost control and the data pipeline behind it.
  • I build the evaluation suite first, so quality is measured rather than argued about.
  • Every feature ships with cost and latency budgets, monitoring, and a defined behaviour for when the model is wrong or unavailable.
  • Model choice is a runtime decision, not an architectural commitment. Integrations are built so a model can be swapped without a rewrite.

Getting an LLM to do something impressive in a notebook takes an afternoon. Getting it to do the same thing reliably, for thousands of users, at a cost that makes sense, with a sensible answer for what happens when it is wrong, that is the work.

This is the work I do on an AI contract, usually alongside the fullstack or data work that feeds it.

What I build

In-product assistants. Chat or command interfaces grounded in a user's own data, with the retrieval layer built properly rather than bolted on.

Retrieval over your content. Search and question-answering across documents, tickets, code or records. Most of the difficulty is in chunking, ranking and freshness rather than in the model.

Structured extraction. Turning unstructured input, documents, emails, transcripts, forms, into validated structured data, with confidence signals and a review path for the cases that need one.

Classification and routing. Triage, tagging and prioritisation where a rules engine has run out of road.

Agentic workflows. Multi-step processes where the system takes actions on a user's behalf, with approval gates on anything consequential.

How I approach it

Evaluation first

Before the feature exists, I build the set of examples that define whether it is working: realistic inputs paired with acceptable outputs, drawn from your actual data.

This sounds like overhead and is the opposite. Without it, every prompt change is an argument between people with different anecdotes. With it, you can change a prompt, swap a model or restructure retrieval and know within minutes whether quality moved.

Budgets set upfront

Cost per request and latency at the ninety-fifth percentile are constraints agreed at the start, alongside the functional requirements, and instrument from the first deployment. A feature that is excellent and unaffordable is not shipped.

Designed for being wrong

Every AI feature gets an explicit answer to "what happens when the output is bad?", validation that catches it, a confidence signal that flags it, a human in the loop for high-stakes cases, or an interaction design that makes correction trivial.

Model as configuration

The provider and model sit behind an interface. When pricing shifts or a better option appears, re-evaluating is a config change and an evaluation run, not a project.

What you get

DeliverableDetail
Shipped featureRunning against real users, behind whatever rollout controls you use
Evaluation suiteVersioned test cases, run in CI, with quality tracked over time
ObservabilityPer-request cost, latency, token usage and quality signals
GuardrailsInput validation, output validation, rate limiting, fallback behaviour
DocumentationHow it works, how to tune it, what to watch, what to do when it degrades

Typical shape

Most integrations run four to ten weeks: roughly a week establishing evaluation and baseline quality, several weeks building and iterating against it, and a final stretch on hardening, cost tuning and rollout.

I usually recommend shipping to a limited cohort first. Real usage surfaces failure modes that no evaluation set anticipates, and it is much cheaper to learn that at five percent of traffic.

Frequently asked questions

What kinds of AI features do you build?

The common ones are in-product assistants and copilots, retrieval over a customer's own documents, structured extraction from unstructured input, classification and routing, natural-language search, and agentic workflows that take multi-step actions on a user's behalf. The pattern matters less than whether the failure modes are acceptable to your users.

How do you know whether an AI feature is actually good?

By measuring it. I build an evaluation set of realistic inputs with known-good outputs, run it on every change, and track quality the way you would track test coverage. Without this, prompt and model changes are guesswork, and teams end up unable to tell an improvement from a regression.

How do you keep inference costs under control?

Cost is a design constraint set at the start, not something discovered in the first month's bill. In practice: routing simple requests to smaller models, caching aggressively where the input repeats, keeping context tight rather than stuffing it, and instrumenting per-request cost so a regression is visible immediately rather than at the end of the billing period.

What happens when the model gets it wrong?

That is a product decision I make with you explicitly rather than discover in production. Depending on the feature, wrong output should be caught by validation, shown with a confidence signal, routed to a human, or made trivially correctable. Features designed as though the model is always right fail badly; features designed for graceful wrongness fail acceptably.

Are we locked into one model provider?

No. I build the integration so the model is a configuration choice behind an interface. Providers change pricing and capability constantly, and being able to re-evaluate and switch without a rewrite is worth the small amount of extra structure it takes.

Start with a 15-minute scoping call

I will tell you honestly whether I am the right fit for what you are building.

See all services↗