Inference Software Engineer

Job summary

Responsibilities

Etched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200.

• Contribute to the architecture and design of the Sohu host software stack
• Implement high-performance, modular code across the complete Etched software stack, consisting of a mix of Rust, C++ and Python.
• Interface with firmware and drivers teams delivering highest-performance HW/SW stack.
• Work with AI model researchers and product-facing teams building out the Etched serving front-end.

What they're looking for

• Experience with C++ and Python
• Familiarity with transformer model architectures and inference serving stacks (vLLM, SGLang, etc.) or experience working in distributed inference/training environments
• Experience working cross-functionally in large software and hardware organizations

Tech stack

• Transformers
• Rust
• C++
• Python
• vLLM
• CUDA