Senior AI R&D Engineer - Performance - Pre-training

Job summary

Responsibilities

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training like our recently released Kolibri-1 model. Our customers - in finance, manufacturing, public administration - need models that understand German, meet European regulatory requirements, and work reliably…

• End-to-End Optimization: Profile training loops using PyTorch Profiler, Nsight Systems and Nsight Compute to identify system- and kernel-level bottlenecks in order to maximize…
• Distributed Strategy and Topology: Configure and tune composite parallelism strategies (e.g. TP, DP, HSDP/FSDP, EP), optimizing load balance, minimizing critical-path…
• Hardware-Aware Modeling: Partner with AI Researchers to define model architectures for hardware efficiency without compromising convergence.

What they're looking for

• Are proficient in Python and the PyTorch library.
• Have a strong engineering background in parallel and/or distributed systems with proven track record of excellence.
• Have hands-on experience with modern machine learning techniques (especially large language models and their life cycle).
• Deeply understand the CUDA programming model.
• Have experience in distributed programming with APIs like NCCL or MPI.

Tech stack

• Python
• PyTorch
• CUDA