AI Engineer - Quality and Post-Training

Job summary

Responsibilities

The bottleneck in building reliable and useful AI agents isn't writing code anymore. It's knowing what good looks like, measuring it, and steadily closing the gap. Glean is building the next generation of enterprise AI assistants and autonomous agents that reason, plan, and act across complex…

• Use eval results and production traces to improve the entire stack, including the agent harness, model routing, prompts, and retrieval, while weighing quality against latency and…
• Work on post-training models with supervised fine-tuning (SFT) and preference- or reward-based optimization (e.g., DPO, PPO, GRPO), including generating the data for training.
• Build RL environments for simulating enterprise workflows, including relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and for internal and external…
• Improve the quality of representative evalsets, LLM judges and agentic verifiers, and calibrate them against human graders so the scores can be trusted in release decisions and…
• Work with product, platform, and research teams across Glean to land quality improvements in production.

What they're looking for

• 5+ years of industry experience in AI or Machine Learning Engineering.
• Hands-on experience post-training LLMs including adapting open-weight models to specialized tasks.
• Experience building evals for AI assistant and agent systems, including generating evalsets, LLM-as-a-judge verifiers, calibrating offline and online signals coupled with good…
• Strong coding skills in Python and/or Go (TypeScript is a plus), proficiency in your ML framework of choice, and a track record of shipping production ML systems.
• Product-minded, able to turn a vague problem into a concrete experiment, and thrive in a tight-knit, cross-functional team.

Tech stack

• Python
• TypeScript