What does an LLM engineer do? Skills, tools and how to get hired

An LLM engineer builds products on top of large language models: assistants, agents, search and document tools. The job is to connect a model to company data and tools, measure whether its output is good, and keep it fast, affordable and reliable in production. LLM engineers rarely train models.

Almost no job ad uses the title "LLM engineer", though. On Tokn, as of 12 October 2026, we count 78 jobs (66 unique roles at 33 companies, about 1 in 10 jobs on the board) that do this work under other names, mostly "AI engineer" and "applied AI engineer". The role is one of the eight paths into an AI career.

In short

• LLM engineers are usually titled AI engineer or applied AI engineer. No ad on Tokn uses the exact title "LLM engineer".
• It is a software engineering job: TypeScript appears in 45% of these ads and PyTorch in 4%, the reverse of ML engineer ads (13% and 61%).
• Agents (at least 70% of roles) and evaluation (at least half) are the core of the work. Fine-tuning shows up in 6%.
• Most people come from software engineering, a PhD is almost never required, and almost a quarter of the roles are fully remote.

What does an LLM engineer do?

An LLM engineer turns a foundation model into a feature people use. The work sits between the model and the product: retrieval, prompts, tools, evaluation and monitoring.

The title took off with Shawn "swyx" Wang's essay The Rise of the AI Engineer, published in June 2023. His point: tasks that once took "5 years and a research team" now "just require API docs and a spare afternoon". He quoted Andrej Karpathy: "One can be quite successful in this role without ever training anything."

LinkedIn's Jobs on the Rise 2026 for Europe put AI engineer first in the UK, France, Italy and Spain and second in Germany, according to Euronews. Two caveats: LinkedIn measures growth in the titles its members hold (January 2023 to July 2025), not job ads, and its "AI engineer" also covers machine learning engineers.

A typical week

Most weeks mix these tasks:

• Product code. Wiring model calls into the product, often across the full stack. Mistral AI wanted someone "comfortable implementing directly in product code, not only notebooks".
• Retrieval (RAG). Embeddings, vector search and reranking over company documents, so the model answers from the right sources.
• Agents and tool use. Letting the model call APIs or run code, then handling state, retries and structured outputs.
• Prompts and context. Anthropic calls this context engineering: "curating and maintaining the optimal set of tokens (information) during LLM inference".
• Evals. Test sets and metrics that show whether a change made the product better or worse.
• Latency, cost and guardrails. Keeping answers fast, affordable and safe. Lemlist wanted someone who looks for "the best quality/latency/cost trade-offs".
• Fine-tuning, occasionally. Doctolib's ad put it plainly: "Establish strong baselines first (prompts, RAG, model selection) before reaching for custom ML."

AI engineer vs ML engineer: what's the difference?

An AI or LLM engineer builds products on top of existing models. A machine learning engineer builds, trains and deploys the models themselves.

Tokn data as of 12 October 2026: 78 AI/LLM engineer jobs (66 unique roles) and the 31 jobs titled machine learning or ML engineer, a narrower set than the one in our ML engineer roadmap. Tokn stores shortened ad text, so theme shares are minimums.

The line blurs: Doctolib and Owkin advertised ML engineer roles focused on LLMs. swyx drew it by data: AI engineers "view product-specific data and evals as their job". If you want to train models yourself, our machine learning engineer roadmap covers that path.

What is the job called? The title zoo

Search for "AI engineer", not "LLM engineer". On Tokn, only 4 engineer titles contain "LLM" at all, and all of them are ML engineer roles.

At model labs, an applied AI engineer often helps customers build on the lab's models: Anthropic's ads asked for "experience in a customer-facing or pre-sales technical role", close to the forward deployed engineer, a role we cover separately. At product companies such as Doctolib, Qonto or Pigment, an AI engineer sits in a product team and ships features.

"Prompt engineer" has almost disappeared as a title: 1 job in 774 on Tokn. To find the roles, browse open AI engineer roles and also search for "agents".

LLM engineer skills: what job ads ask for

Employers want a strong Python engineer, often with TypeScript, who has shipped LLM features with agents and evals. Frameworks matter less than you might think.

The tech stack

Front-end skills (React, TypeScript) come up often because many of these roles are full stack. You don't need every framework: Anthropic's guide to building effective agents recommends "finding the simplest solution possible, and only increasing complexity when needed", starting with direct API calls. For the ranking across all AI jobs, see the tools AI job postings ask for.

The skills that matter more than any framework

Based on 66 unique roles (733 board-wide). Fine-tuning is rarer here than across Tokn as a whole.

Evals are the signature skill. Explicit "evals" wording appears in a quarter of these roles, four times the rate across Tokn. Elastic asked for "a track record of eval-driven product improvement: you have diagnosed agent or LLM behaviour from traces and user feedback".

How do you become an LLM engineer?

Most LLM engineers come from software engineering. What gets you hired is proof that you shipped an LLM feature to real users and measured it.

The usual route: from software engineering

55% of the roles name software engineering experience. Mistral AI's AI engineer ad said "backgrounds that fit well include ML engineers moving closer to product, or software engineers with real AI/ML production experience." Anthropic asked its applied AI engineers for "4+ years of experience as a Software Engineer, Forward Deployed Engineer, or technical founder".

Experience and degrees

31 of the 66 roles state a minimum. The median is 4 years and most ask for 3 to 6. A PhD shows up in 3 of 48 AI engineer ads that list requirements, against 24 of 63 research jobs. Pigment's ad sums up the norm: a degree "or experience writing production-grade code".

Build proof, not demos

Inato asked for "1 year in a production environment (not a pet project)". If you are switching from software engineering, build the evidence:

1. Ship one LLM feature end to end: RAG over real documents, or an agent that calls real tools.
2. Write an eval set with typical, edge and adversarial cases, score it automatically, and show the before and after numbers.
3. Track latency and cost per request, and show what you did to bring them down.
4. Publish the code and a short write-up of what failed and what you changed.

What to read

• Chip Huyen, AI Engineering: Building Applications with Foundation Models (O'Reilly, January 2025), strong on evaluation.
• Anthropic's "Building effective agents" (December 2024) and "Effective context engineering for AI agents" (September 2025), linked above.
• Anthropic's guide to building evaluations and OpenAI's evaluation best practices. Read OpenAI's for the method: its hosted Evals platform is scheduled to shut down on 30 November 2026.
• Hugging Face's free Agents Course and LLM Course.

Entry-level options

Junior openings are rare. Six internships were advertised in October, at Veepee, Equativ, Shift Technology, Ekimetrics and Celonis. Our step-by-step guide to landing an AI job covers the first move.

Where are the LLM engineer jobs in Europe?

Paris and London lead, with remote close behind. Of the 78 jobs, roughly 20 each are in Paris and London and 18 are fully remote, followed by Madrid and Munich.

OpenAI, Anthropic and Mistral AI hire on-site or hybrid: none of their roles here was fully remote. Overall, 23% of these jobs are fully remote, against about one in six across Tokn. Remote-friendly employers include ClickHouse, Pigment, Alan, GitLab and Qonto.

Tokn files most AI engineer roles under machine learning, so the live list below mixes them with ML engineer jobs:

Too few of these ads list a salary for a reliable figure. For pay, see what AI roles pay in Europe.

Find LLM engineer jobs on Tokn

• Open AI engineer roles, then a second search for "agents".
• Remote AI jobs.
• AI jobs in Paris and AI jobs in London, the two biggest hubs.

Create a job alert for "AI engineer" to get new roles by email.

FAQ

Is an LLM engineer the same as an AI engineer?

In practice, yes. "LLM engineer" is what people search for, while ads say "AI engineer", "applied AI engineer" or "software engineer, agents". No job on Tokn uses the exact title.

Do LLM engineers train or fine-tune models?

Rarely. Fine-tuning or post-training appears in 6% of AI engineer roles on Tokn, against 19% of ML engineer roles. Most of the work uses existing models through APIs.

Do I need a PhD or a machine learning degree?

No. A PhD shows up in 3 of 48 AI engineer ads that list requirements, and ads often accept experience instead of a degree. PhDs are common in research jobs, not this one.

Is prompt engineering still a job?

Barely, as a title: 1 job in 774 on Tokn. As a skill, it lives inside the AI engineer role, where about a quarter of ads mention prompts. The wider discipline is now called context engineering.

Is LLM engineering a good career in 2026?

The signals are strong. LinkedIn's Jobs on the Rise 2026 ranked AI engineer the fastest-growing job title in the UK, France, Italy and Spain, though its definition includes ML engineers. On Tokn, these roles make up about 1 in 10 jobs, at 33 companies.