Research Engineer, LangSmith Engine

Job summary

Responsibilities

We’re looking for an experienced research engineer to help make the Engine agent more capable and more efficient.

• Build and maintain benchmarks and evaluations that measure the quality and efficiency of Engine agents on real-world tasks.
• Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and agent strategies.
• Explore and implement post-training and fine-tuning techniques when they can meaningfully improve agent capabilities, quality, or cost.
• Turn successful experiments into production improvements, working closely with engineers and researchers to measure impact and prevent regressions.
• Help define the ML roadmap and technical direction for improving Engine agents, and mentor other engineers through strong technical leadership.

What they're looking for

• 4+ years of experience in ML/AI research, or a closely related field.
• Master’s or Ph.D. in a relevant scientific field.
• Hands-on experience working with LLMs and AI agents, including analyzing model behavior and improving real-world performance
• Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems; you know how to tell whether a change actually made an agent better.
• Strong software engineering skills, with a track record of taking ideas from research prototype to measurable production impact.

Tech stack

• LangGraph