Senior Software Engineer (MLOps) – Serving

Who you are

You have 5+ years of experience in software engineering with a focus on machine learning operations — you've successfully designed and implemented systems that serve ML models at scale, ensuring high availability and performance. Your expertise in Python allows you to develop robust solutions that meet the demands of production environments.

You possess a deep understanding of CI/CD practices — you've built and maintained pipelines that facilitate the seamless deployment of machine learning models, enabling teams to iterate quickly and efficiently. Your experience with Ray demonstrates your ability to optimize inference infrastructure for both low- and high-throughput workloads.

You are familiar with observability and reliability principles — you understand the importance of monitoring and logging in production systems, and you have implemented strategies to ensure that deployed models perform as expected under various conditions. Your collaborative nature allows you to work effectively with applied scientists and product teams to deliver impactful solutions.

You are passionate about enabling self-service tools for model deployment — you believe in empowering teams to manage their own workflows, and you have experience creating user-friendly interfaces that simplify the deployment and testing of models. Your proactive approach to continuous improvement drives you to seek out opportunities for enhancing existing processes.

Desirable

Experience with A/B testing and rollback strategies is a plus — you understand the importance of validating model performance in production and have implemented techniques to ensure safe deployments. Familiarity with cloud platforms and distributed systems will further enhance your ability to contribute to our infrastructure.

What you'll do

As a Senior MLOps Engineer at Datadog, you will architect and build systems for serving machine learning and large language models across multiple data centers — your work will ensure that our models are deployed reliably and efficiently, meeting strong service level agreements. You will design and optimize Ray-based inference infrastructure, enabling high-performance model serving that can handle varying workloads.

You will collaborate closely with applied scientists to enable them to deploy and test their models using self-service tools — your expertise in CI/CD will be crucial in streamlining these processes, allowing for rapid iteration and deployment. You will also implement observability practices to monitor model performance and reliability, ensuring that any issues are quickly identified and addressed.

Your role will involve working across infrastructure, observability, and machine learning engineering domains — you will be a key player in delivering production-ready inference capabilities that are used by teams across Datadog. You will contribute to the continuous improvement of our model serving systems, exploring new technologies and methodologies to enhance our capabilities.

What we offer

At Datadog, we value our office culture and the relationships that it fosters — we operate as a hybrid workplace, allowing you to create a work-life harmony that best fits your needs. You will have the opportunity to work with a talented team of engineers and scientists who are passionate about building innovative solutions in the AI space.

We encourage you to apply even if your experience doesn't match every requirement — we believe that diverse teams build better products, and we are committed to creating an inclusive environment where everyone can thrive. Join us in shaping the future of observability and security products powered by machine learning.