Data Scientist/ Engineer (Hybrid -Boston)

Job summary

Responsibilities

Shift delivers AI agents that transform insurers' most critical work. By combining deep industry expertise and unmatched data resources, Shift provides proven results that have earned the trust of hundreds of the world's leading insurers.

• Your role will be to actively contribute to the US- Insurance roadmap and clients, and working on various data types such as structured data, free text, documents and images.
• Build and productionize data pipelines (structured, text, documents, images) optimized for LLMs and multi-modal models.
• Design, develop and deploy LLM-based solutions (RAG, embeddings, instruction tuning) for subrogation, claims handling, document understanding, and related use cases.
• Experiment with the latest in Agentic AI technologies (Langchain/Langgraph, OpenAI Agent SDK, MCP, A2A) and develop MVP for the next generation of autonomous subrogation solutions.
• Develop "Chain-of-Thought" and "ReAct" prompting strategies to ensure the agent can justify its liability percentages based on the Comparative Negligence laws of different…

What they're looking for

• Expert proficiency in production-level object-oriented programming (OOP) for building scalable and reliable systems.
• Proven hands-on experience with Large Language Models (LLMs) and generative AI techniques (including RAG, embeddings, prompt engineering, and model tuning), leveraging frameworks…
• Solid foundation in ML fundamentals with practical experience in the full machine learning lifecycle, including model evaluation, monitoring, versioning, and deployment in…
• Experience designing and implementing robust data pipelines for document, OCR, and multi-modal data workflows.
• Agentic Frameworks: Experience with integrating frameworks like LangChain/LangGraph, OpenAI Agent SDK, CrewAI, A2A/MCP with Databricks or Azure-hosted models (e.g., DBRX, OpenAI…

Tech stack

• RAG
• LangChain
• LangGraph
• Databricks
• Azure
• OpenAI API
• Spark
• MLflow