Data Engineer

Who you are

You have a strong background in data engineering, with experience in building and maintaining core datasets that include vessels characteristics, companies, and geospatial data. You are skilled in creating and maintaining REST APIs, and you have a solid understanding of streaming pipelines, particularly with Kafka. Your expertise in Apache Spark allows you to efficiently handle batch processing tasks, ensuring data is processed accurately and in a timely manner.

You take pride in your end-to-end ownership of development tasks, starting with a thorough understanding of assigned tickets and requirements. You are adept at designing and building functionality, including APIs and data processing components, and you ensure that your code is deployed to development environments and undergoes rigorous peer and product testing. Your attention to detail ensures that all code is compliant with defined standards and best practices.

You are committed to writing and executing unit, integration, and functional tests that align with defined test scenarios. You understand the importance of validation and compliance in your work, and you strive to maintain high standards in all aspects of your role. After release, you take responsibility for monitoring system performance, alerts, and service level objectives (SLOs) to ensure optimal functionality and reliability.

Desirable

Experience with cloud platforms and data warehousing solutions would be a plus, as would familiarity with data visualization tools. You are a proactive communicator, able to collaborate effectively with cross-functional teams to deliver impactful results.

What you'll do

In this role, you will be responsible for building and maintaining Kpler's core datasets, which are crucial for providing valuable insights to clients in the commodities, energy, and maritime sectors. You will create and maintain REST APIs that facilitate data access and integration, ensuring that clients can easily navigate complex markets.

You will develop streaming pipelines using Kafka to handle real-time data processing, as well as batch pipelines with Apache Spark for more extensive data operations. Your work will involve end-to-end ownership of development tasks, from understanding requirements to deploying code and conducting thorough testing.

You will collaborate closely with other engineers and stakeholders to ensure that the data processing components you build meet the needs of the organization and its clients. Your role will also include monitoring system performance and responding to alerts to maintain optimal functionality.

What we offer

At Kpler, you will join a team of over 700 experts from more than 35 countries, all dedicated to transforming intricate data into actionable strategies. We provide a supportive environment where you can leverage cutting-edge innovation for impactful results. You will have the opportunity to work on user-friendly platforms that simplify global trade information and empower organizations to make informed decisions.

We encourage you to apply even if your experience doesn't match every requirement. Join us in our mission to deliver top-tier intelligence and help our clients stay ahead in a dynamic market landscape.