Staff AI & Agentic Platform Engineer

Job summary

Responsibilities

Discovery is building an expeditionary force: a team of engineers who want to solve difficult problems and who navigate unfamiliar territory as an operating standard. As an AI & Agentic Platform Engineer, you pick up a concept nobody has proven and build the analysis or prototype that tests it to…

• Develop Test Scenarios Simulation Environments: Develop comprehensive test scenarios, and simulation environments to assess agentic AI performance in classified simulations.
• Validate AI on Classified & Edge Platforms: Lead the integration and validation of agentic AI systems onto classified platforms and edge hardware.
• Build Reusable Evaluation Pipelines: Build reusable evaluation pipelines, automated test harnesses, and monitoring dashboards for continuous validation.
• Define Actionable Metrics: Figure out what good metrics look like for AI models and traditional models—establishing systematic, historic capture of performance rather than…
• Partner Cross-Functionally: Partner with cross-functional teams to define requirements, document test results, and contribute to AI model cards and deployment readiness reviews.

What they're looking for

• Technical Expertise: Bachelor's or Master's degree in Computer Science, Machine Learning, Software Engineering, Computer Engineering, Electrical Engineering, or a related…
• Programming Proficiency: At least 8-10 years of hands-on experience writing production-grade code, with strong Python skills for building evaluation pipelines, test harnesses…
• Evaluation Ownership Mindset: Demonstrated experience owning evaluation or automated testing for production ML or software systems—test scenarios, metrics, regression suites, and…
• AI/ML Evaluation Experience: Hands-on experience designing evaluation methodologies for AI or machine learning models—defining metrics, building benchmarks, and assessing model…
• Simulation Experience: Experience designing or working extensively with simulation environments to exercise model and system behavior.

Tech stack

• Python
• Go
• C++