Senior Software Engineer, Data Pipelines

Who you are

You have 5+ years of experience in software engineering, particularly in building and maintaining large-scale distributed systems. Your background includes a strong understanding of data pipelines and the technologies that support them, enabling you to effectively manage and optimize data flow across various services. You are proficient in Python and have experience with tools like Apache Kafka and AWS, which are essential for handling data at scale. Your familiarity with containerization technologies such as Docker allows you to streamline deployment processes and enhance system reliability. You understand the importance of monitoring and analyzing system performance metrics, and you are skilled in using tools like Elasticsearch and PostgreSQL to ensure data integrity and availability.

You thrive in collaborative environments and enjoy working closely with cross-functional teams to deliver impactful solutions. Your problem-solving skills are complemented by your ability to communicate complex technical concepts to non-technical stakeholders, ensuring alignment and understanding across the organization. You are committed to continuous learning and improvement, always seeking opportunities to enhance your skills and contribute to the team's success. We encourage you to apply even if your experience doesn't match every requirement.

Desirable

Experience with additional data processing frameworks or languages, such as Apache Spark or Scala, would be a plus. Familiarity with CI/CD practices and tools can also enhance your contributions to the team.

What you'll do

As a Senior Software Engineer on the Data Pipelines team at Lyft, you will play a crucial role in maintaining and optimizing the infrastructure that supports critical data flow across the organization. You will design and implement robust data pipelines that ensure the seamless transfer of information between various services, impacting millions of drivers and passengers daily. Your responsibilities will include monitoring system performance, analyzing metrics, and identifying areas for improvement to enhance the overall efficiency of our data operations.

You will collaborate with other engineers to develop automated solutions for infrastructure management, ensuring that our systems are reliable and scalable. Your expertise in Python and Apache Kafka will be instrumental in building data processing workflows that meet the needs of internal customers. You will also engage in troubleshooting and resolving issues that arise within the data pipeline, leveraging your analytical skills to diagnose problems and implement effective solutions.

In addition to your technical contributions, you will have the opportunity to mentor junior engineers, sharing your knowledge and best practices to foster a culture of learning within the team. You will participate in design reviews and contribute to architectural discussions, helping to shape the future of our data infrastructure.

What we offer

At Lyft, we value a diverse and inclusive work environment where all team members can thrive. We offer competitive compensation packages, including salary and benefits that reflect your experience and contributions. You will have access to professional development opportunities, allowing you to grow your skills and advance your career within the company. Join us in our mission to improve transportation experiences for millions of users while working with cutting-edge technologies in a supportive and collaborative team.