Senior IA Inference Engineer

Job summary

Responsibilities

We are looking for a Senior Inference Engineer to join the ML Platform (TIGER) team in AI & Machine Learning.

• Build the shared LLM serving foundations with MLOps engineers — define the serving stack, GPU capacity planning, and deployment automation for open-weight models (Gemma, Mistral…
• Own the model registry and promotion path — establish clear processes for versioning, validating, and promoting models into production, benchmark performance and output quality…
• Optimize inference performance and efficiency — operate and tune serving engines (vLLM, Triton Inference Server, TensorRT-LLM) to reduce time-to-first-token, inter-token latency…
• Own production reliability and disaster recovery — define SLOs, run load tests before launches, implement safe rollouts, rollbacks, and autoscaling, and participate in on-call…
• Make serving performance and costs measurable — build Datadog dashboards, alerting, and cost reporting to track latency, GPU utilization, and inference spend, and identify…

Tech stack

• Kubernetes
• vLLM
• Triton
• Python
• Terraform
• Ray
• CUDA
• C++