AI Infrastructure Engineer

Job summary

Responsibilities

Mercor is creating a new category of work where expertise powers AI advancement. Achieving this requires an ambitious, fast-paced and deeply committed team. You’ll work alongside researchers, operators, and AI companies at the forefront of shaping the systems that are redefining society.

• Design, deploy and operate GPU clusters across AWS, GCP, and other hyperscalers/neoclouds/on-premise data centers, including custom Kubernetes and Slurm operators and the…
• Build scheduling that keeps accelerators busy: gang scheduling for distributed jobs, bin-packing to reduce fragmentation, and quotas and fair share for multi-tenant teams.
• Run distributed training infrastructure on PyTorch Distributed, Ray, or Megatron, with fast checkpointing and model loading.
• Deploy and tune high-throughput LLM serving with vLLM or similar: tensor and model/expert parallelism, prefill/decode disaggregation, KV cache offload, FP8/FP4 quantization.
• Build benchmarking and traffic-replay tools that measure TTFT, TPOT and latency percentiles, and set per-config performance baselines.

Tech stack

• Kubernetes
• vLLM
• AWS
• GCP
• PyTorch
• Ray
• Python
• Go