Senior Data Platform Engineer

United States Digital Space LLC; Amsterdam; On-site; EUR 90,000 - 140,000; Full time

Job summary

United States Digital Space LLC in Amsterdam is seeking a senior data engineer to build and enhance scalable data platforms. You will develop streaming and batch pipelines, and collaborate with analysts, data scientists and engineers to deliver high-quality tooling and data services.

The role emphasizes in-person collaboration at our Amsterdam office with no remote-only options. A strong background in Python and big data frameworks is required.

Qualifications

• 6+ years of Python object oriented programming experience.
• Experience with distributed data and compute systems such as Spark, Trino, Druid.
• Experience developing and maintaining DevOps pipelines and development ecosystems.
• Real-time and batch data pipelines using Kafka and Spark streaming.
• Familiarity with Kubernetes, Docker and Hadoop ecosystems.

Responsibilities

• Build and maintain scalable, high-performance big data platforms.
• Improve tooling to enhance data discoverability and metadata presentation.
• Develop streaming processing applications and frameworks.
• Collaborate with analysts, data scientists and data engineers; coordinate with security and infra teams.

Skills

Team player

Strong communication

Stakeholder management

Tools

Python

Spark

Trino

Druid

Kafka

Kubernetes

Hadoop

Job description

What you’ll do

You’ll be asked to both build new tools and improve the existing tooling. This role is perfect for you if you're passionate about few of the following:

• Develop and maintain scalable and high performance big data platforms.

• Work with distributed systems in all shapes and flavors (databases, filesystems, compute, etc.)

• Identify opportunities to improve continuous release and deployment environments

• Build or deploy tools to enhance data discoverability through the collection and presentation of metadata

• Introduce and extend tools to enhance the quality of our data, platform-wide

• Explore and introduce technologies and practices to reduce the time to insight for analysts and data scientists

• Develop streaming processing applications and frameworks

• Experience in infrastructure and large-scale private cloud systems

Who you are

• At least 6+ years experience with Python object oriented programming.

• Team player with strong communication skills

• Ability to work closely with diverse stakeholders you enable (analysts, data scientists, data engineers, etc.) and depend upon (infrastructure, security, etc).

• Experience developing and maintaining:

• Distributed data and compute systems like Spark, Trino, Druid, etc

• Data modelling tooling

• DevOps pipelines and development ecosystems.

• Real-time and batch data pipelines (via Kafka, Spark streaming) with an eye for frameworks, and emphasis on user friendliness and quality.

• Kubernetes ecosystem (k8s, docker), and/or Hadoop ecosystems (Hive, Yarn, HDFS, Kerberos).

Our Diversity, Equity and Inclusion commitments

Our unique approach is a product of our diverse perspectives. This diversity of backgrounds and cultures is essential in helping us maintain our momentum. Our business and technical challenges are unique, and we need as many different voices as possible to join us in solving them - voices like yours. No matter who you are or where you’re from, we welcome you to be your true self at the company.

Studies show that women and members of underrepresented communities apply for jobs only if they meet 100% of the qualifications. Does this sound like you?

What’s next?

Ensuring a smooth and enjoyable candidate experience is critical for us. We aim to get back to you regarding your application within 5 business days. Our interview process tends to take about 4 weeks to complete, but may fluctuate depending on the role. Learn more about our hiring process here. Don’t be afraid to let us know if you need more flexibility.

This role is based out of our Amsterdam office. We are an office-first company and value in-person collaboration; we do not offer remote-only roles.