Member of Technical Staff, Data Analysis and Evaluation

Job summary

Responsibilities

As a Member of Technical Staff in Data Analysis and Evaluation, you will play a pivotal role in ensuring the quality, reliability, and performance of our large language models (LLMs).

• Design and oversee data collection tasks, including supporting human annotators and ensuring data quality.
• Develop and apply statistical methods to evaluate the quality and reliability of datasets.
• Analyse and assess the generalisability and robustness of ML systems across diverse use cases.
• Collaborate with teams to improve dataset quality and model performance.
• Train and fine-tune large language models (LLMs) on distributed training infrastructures.

What they're looking for

• Extremely strong software engineering skills.
• Strong expertise in designing and conducting data collection tasks, including working with human annotators.
• Strong statistical skills and experience evaluating scientific experiments related to data collection and model performance.
• Experience analysing datasets with respect to their quality, biases, and suitability for training ML models.
• Hands-on experience training large language models (LLMs) on distributed training infrastructures.

Tech stack

• Python
• PyTorch
• TensorFlow
• JAX