Lead Data Engineer

Who you are

You have 5+ years of experience in data engineering, with a strong background in developing data architecture and solutions that meet business objectives. Your expertise in Python and SQL allows you to create intricate data processing pipelines that address complex challenges. You are familiar with cloud platforms like AWS and have experience with tools such as Airflow for orchestrating data workflows. You understand the importance of data quality and are skilled in collaborating with stakeholders to leverage data effectively. Your hands-on coding experience is complemented by your ability to lead teams and manage projects from inception to completion.

You are comfortable working in a collaborative environment, where you can guide and mentor junior engineers while also engaging with clients to understand their strategic objectives. Your experience includes designing scalable implementations of data models and ensuring that solutions are delivered on time through effective project management. You are committed to continuous learning and staying updated with the latest trends in data engineering and architecture.

Desirable

Experience with Docker and PostgreSQL is a plus, as these tools are essential for building and deploying data solutions. Familiarity with data quality frameworks and best practices will enhance your ability to deliver high-quality results. You may also have experience with other cloud services and data processing technologies that can contribute to the success of your projects.

What you'll do

In this role, you will lead and manage data engineering projects, ensuring that they align with client objectives and are delivered on time. You will collaborate with cross-functional teams to define project goals and scope, and you will oversee the technical design and implementation of data solutions. Your responsibilities will include creating and managing data pipelines, working closely with data scientists to implement their models, and advising clients on best practices for data usage.

You will be responsible for writing clean, iterative code based on test-driven development (TDD) principles, and you will leverage continuous delivery practices to support and operate data pipelines. Your role will also involve conducting technical reviews and providing insights to clients on how to utilize distributed storage and computing effectively. You will play a key role in shaping the data strategy for various clients, helping them to unlock the potential of their data assets.

What we offer

At ThoughtWorks, we foster a culture of innovation and collaboration, where your contributions will have a meaningful impact on our clients' success. We provide opportunities for professional growth and continuous learning, encouraging you to expand your skills and expertise in data engineering. You will work alongside talented professionals who are passionate about technology and its ability to solve complex business problems. We offer competitive compensation and benefits, along with a supportive work environment that values diversity and inclusion.