Senior AI Researcher- Pre-training

Job summary

Responsibilities

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training. Our customers — in finance, manufacturing, and public administration — need models that understand German, meet European regulatory requirements, and work reliably in high-stakes settings.

• Recipe & Architecture Optimization: Own core elements of the training recipe (optimizers, schedules, initialization) and design PyTorch-based architectural improvements to…
• Scaling Strategy & Predictability: Develop hyperparameter scaling laws and scale-up methodologies, using small-scale proxy experiments to reliably predict multi-thousand-GPU…
• Stability, Diagnostics & Debugging: Investigate complex convergence issues (loss spikes, divergence) and resolve hard-to-reproduce distributed system failures like communication…
• System-Model Co-Design: Partner with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and…

What they're looking for

• You are proficient in Python and deeply familiar with PyTorch-based training workflows.
• You have a strong track record in machine learning research and software engineering, demonstrated through shipped models, impactful open-source contributions, or published…
• You have a strong mathematical foundation and are comfortable reasoning formally about optimisation, scaling behaviour, and training dynamics.
• You deeply understand transformer training dynamics, optimisation, and the behaviour of large distributed training jobs.
• You can design rigorous experiments, reason clearly from noisy results, and translate empirical observations into robust training decisions.

Tech stack

• PyTorch
• Python