Sr Lead Machine Learning Engineer

Who you are

You have extensive experience in machine learning and AI, with a strong background in designing and implementing evaluation frameworks that reflect real-world task success. Your expertise in Python and familiarity with machine learning libraries such as TensorFlow and PyTorch enable you to craft rigorous evaluation methodologies that ensure quality and trust in AI systems. You understand the importance of data analysis and can assess outcomes beyond traditional accuracy metrics, focusing on what truly works for users.

You thrive in collaborative environments, working cross-functionally with various teams to evaluate human+AI interactions. Your ability to communicate complex concepts clearly allows you to engage effectively with stakeholders, ensuring that the evaluation frameworks you develop align with business goals and user needs. You are passionate about revolutionizing agent evaluation and making a measurable impact on AI systems that power the future of work.

Desirable

Experience with large-scale AI systems and a deep understanding of the ethical implications of AI technology are highly valued. Familiarity with agile methodologies and experience in leading projects will set you apart as a candidate. You are also encouraged to bring innovative ideas to the table, contributing to the continuous improvement of evaluation processes.

What you'll do

In this role, you will be at the forefront of defining how success is measured for AI agents performing real-world tasks. You will design and implement comprehensive evaluation frameworks that reflect real-world task success for agentic systems. Your work will involve assessing the effectiveness of human+AI collaboration and uncovering insights that go beyond traditional metrics. You will collaborate with product teams to ensure that the evaluation frameworks you create are integrated into the development process, shaping the features and functionalities of AI agents.

You will also be responsible for analyzing data from various sources to inform your evaluation frameworks, ensuring that they are robust and applicable to real-world scenarios. Your insights will help drive business transformation by enabling Upwork to leverage AI-powered work solutions effectively. You will work closely with engineers, data scientists, and product managers to ensure that the evaluation processes are aligned with the overall goals of the organization.

What we offer

At Upwork, you will be part of a mission-driven organization that values innovation and collaboration. We offer a competitive salary range of $195,000—$308,000 USD, along with eligibility for an annual bonus plan and participation in our long-term equity incentive program. You will have the opportunity to work remotely, allowing for flexibility in your work environment. Join us in revolutionizing the way AI systems are evaluated and make a significant impact on the future of work.