Senior | Staff Software Engineer - AI / ML

Job summary

Responsibilities

Frontier AI data is expensive to make and hard to measure. Every task we deliver is tested against the strongest models, often through many long-running agent rollouts. Your job is to make that process faster, cheaper, and more rigorous with ML and AI

• Efficient agentic evals. Cut the cost of long-horizon agent evaluation with adaptive sampling, statistically grounded early stopping, model cascades, caching, and cheap-first…
• AI model routing. Route every eval and judge call to the cheapest model that clears the quality bar, with fallback, monitoring, and cost attribution.
• Fine-tuned small models. Fine-tune and serve open-weight models (LoRA and other parameter-efficient methods) where they match frontier quality, and know when they don't.
• Predictive difficulty. Build models that estimate how hard a task is for frontier systems before running a single rollout.
• Measurement for AI data. Build golden datasets, quantify the accuracy and calibration of LLM-as-judge systems, and make quality reproducible across projects.

What they're looking for

• 5+ years building production ML or software systems, with end-to-end ownership from prototype to production
• Hands-on experience running LLM or ML workloads in production, and comfort reasoning about non-deterministic systems
• Deep grounding in statistics and experimentation: experiment design, hypothesis testing, sampling, and confidence intervals
• Strong Python and software engineering fundamentals, including testing, code review, and system design
• Experience designing evaluations and interpreting results rigorously

Tech stack

• Python