Principal AI Engineer - Context - Agents and Context

Job summary

Responsibilities

The Context Engine team builds the knowledge layer that AI agents use to work with enterprise data in Elasticsearch. We extract knowledge from any data sources into a structured AI Index, serve it to agents through public APIs, MCP tools and framework integrations, and close the loop with agent…

• Own the production improvement loop for Context Engine: understand how extraction automations, retrieval tools and memory behave, based on offline evaluations and customer…
• Define how we iterate on agents and skills safely: versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and the…
• Design the telemetry we need to make data-informed engineering decisions: what to capture from agent traces, tool calls and knowledge retrieval, how it lands in Elasticsearch…
• Partner with the data science team on evaluation strategy: golden datasets, evaluators to gate on quality, latency and cost.
• Raise the bar across the team: review designs and PRs, mentor engineers in eval-driven development, and write the technical proposals that shape the roadmap.

What they're looking for

• 10+ years of software engineering experience, with the recent years spent shipping and operating AI-driven products on real production traffic, ideally products with public APIs…
• A track record of eval-driven product improvement: you have diagnosed agent or LLM behaviour from traces and user feedback, designed the evaluation that exposed the problem…
• Direct experience building agents with state and memory, and iterating on prompts, skills and tool behaviour safely in production.
• Familiarity with MCP, including exposing public MCP servers and tools.
• Experience designing telemetry for AI systems, and using it to make engineering and product decisions.

Tech stack

• LangChain
• TypeScript
• Python
• LangGraph