LLM features wired into real products and real data
AI integration
Contract AI engineering for production products: retrieval over your documents, structured extraction, assistants and agent workflows, with the evaluation suite, guardrails and cost controls needed to run them for real users.
Key takeaways
- The engineering difficulty in AI features is rarely the model call. It is evaluation, failure handling, cost control and the data pipeline behind it.
- I build the evaluation suite first, so quality is measured rather than argued about.
- Every feature ships with cost and latency budgets, monitoring, and a defined behaviour for when the model is wrong or unavailable.
- Model choice is a runtime decision, not an architectural commitment. Integrations are built so a model can be swapped without a rewrite.
Getting an LLM to do something impressive in a notebook takes an afternoon. Getting it to do the same thing reliably, for thousands of users, at a cost that makes sense, with a sensible answer for what happens when it is wrong, that is the work.
This is the work I do on an AI contract, usually alongside the fullstack or data work that feeds it.
What I build
In-product assistants. Chat or command interfaces grounded in a user's own data, with the retrieval layer built properly rather than bolted on.
Retrieval over your content. Search and question-answering across documents, tickets, code or records. Most of the difficulty is in chunking, ranking and freshness rather than in the model.
Structured extraction. Turning unstructured input, documents, emails, transcripts, forms, into validated structured data, with confidence signals and a review path for the cases that need one.
Classification and routing. Triage, tagging and prioritisation where a rules engine has run out of road.
Agentic workflows. Multi-step processes where the system takes actions on a user's behalf, with approval gates on anything consequential.
How I approach it
Evaluation first
Before the feature exists, I build the set of examples that define whether it is working: realistic inputs paired with acceptable outputs, drawn from your actual data.
This sounds like overhead and is the opposite. Without it, every prompt change is an argument between people with different anecdotes. With it, you can change a prompt, swap a model or restructure retrieval and know within minutes whether quality moved.
Budgets set upfront
Cost per request and latency at the ninety-fifth percentile are constraints agreed at the start, alongside the functional requirements, and instrument from the first deployment. A feature that is excellent and unaffordable is not shipped.
Designed for being wrong
Every AI feature gets an explicit answer to "what happens when the output is bad?", validation that catches it, a confidence signal that flags it, a human in the loop for high-stakes cases, or an interaction design that makes correction trivial.
Model as configuration
The provider and model sit behind an interface. When pricing shifts or a better option appears, re-evaluating is a config change and an evaluation run, not a project.
What you get
| Deliverable | Detail |
|---|---|
| Shipped feature | Running against real users, behind whatever rollout controls you use |
| Evaluation suite | Versioned test cases, run in CI, with quality tracked over time |
| Observability | Per-request cost, latency, token usage and quality signals |
| Guardrails | Input validation, output validation, rate limiting, fallback behaviour |
| Documentation | How it works, how to tune it, what to watch, what to do when it degrades |
Typical shape
Most integrations run four to ten weeks: roughly a week establishing evaluation and baseline quality, several weeks building and iterating against it, and a final stretch on hardening, cost tuning and rollout.
I usually recommend shipping to a limited cohort first. Real usage surfaces failure modes that no evaluation set anticipates, and it is much cheaper to learn that at five percent of traffic.
Frequently asked questions
What kinds of AI features do you build?
The common ones are in-product assistants and copilots, retrieval over a customer's own documents, structured extraction from unstructured input, classification and routing, natural-language search, and agentic workflows that take multi-step actions on a user's behalf. The pattern matters less than whether the failure modes are acceptable to your users.
How do you know whether an AI feature is actually good?
By measuring it. I build an evaluation set of realistic inputs with known-good outputs, run it on every change, and track quality the way you would track test coverage. Without this, prompt and model changes are guesswork, and teams end up unable to tell an improvement from a regression.
How do you keep inference costs under control?
Cost is a design constraint set at the start, not something discovered in the first month's bill. In practice: routing simple requests to smaller models, caching aggressively where the input repeats, keeping context tight rather than stuffing it, and instrumenting per-request cost so a regression is visible immediately rather than at the end of the billing period.
What happens when the model gets it wrong?
That is a product decision I make with you explicitly rather than discover in production. Depending on the feature, wrong output should be caught by validation, shown with a confidence signal, routed to a human, or made trivially correctable. Features designed as though the model is always right fail badly; features designed for graceful wrongness fail acceptably.
Are we locked into one model provider?
No. I build the integration so the model is a configuration choice behind an interface. Providers change pricing and capability constantly, and being able to re-evaluate and switch without a rewrite is worth the small amount of extra structure it takes.
Related resources
Senior fullstack engineering
Contract senior fullstack engineering for product teams. TypeScript, React and Next.js on the front, Node, Python and Java services behind, Postgres and cloud infrastructure underneath, delivered with tests, CI/CD and observability.
Senior data engineering
Contract senior data engineering: medallion lakehouses on Microsoft Fabric, Databricks and Snowflake, Spark and dbt pipelines, data quality checks and the modelled gold layers that finance, operations and ML teams build on.
Why AI features take five times longer than the demo suggested
The prototype took two days and was genuinely impressive. Then it took four months to ship. Here is where that time goes, why it is not anyone's fault, and how to plan for it.
Start with a 15-minute scoping call
I will tell you honestly whether I am the right fit for what you are building.