Senior Data Engineer

Spotify; New York (NY); Remote; USD 160,000 - 229,000; Full time

Job summary

Join a forward-thinking company as a Senior Data Engineer, where you will play a vital role in enhancing Spotify's advertising technology. This exciting position involves building scalable data pipelines, collaborating with cross-functional teams, and ensuring the efficiency of critical reporting systems. With a commitment to innovation and inclusivity, you will contribute to shaping the future of audio streaming while working in a flexible environment that values your unique perspective. If you're passionate about data and want to make a significant impact, this opportunity is perfect for you.

Benefits

Health Insurance

Paid Parental Leave

401(k) Retirement Plan

Monthly Meal Allowance

Paid Days Off

Paid Flexible Holidays

Paid Sick Leave

Qualifications

• Experience with high-volume data in distributed systems.
• Strong programming skills in Python, Java, or Scala.
• Hands-on experience with orchestration tools like Airflow.

Responsibilities

• Maintain and enhance reporting pipelines ensuring they remain business-critical.
• Build large-scale batch and real-time data pipelines.
• Collaborate with stakeholders to translate business requirements into technical solutions.

Skills

Python

Java

Scala

Hadoop

BigTable

Cassandra

GCP

AWS

Azure

Data Processing Frameworks

Tools

Airflow

GCP Dataflow

AWS EMR

Apache Beam

Job description

We are looking for a Senior Data Engineer to join our Advertising Product & Technology organization. Our teams are charged with scaling Spotify's Free Product and our Podcast Monetization Mission. You will work directly on an array of product features, advertiser publishing tools, and insights and reporting functionalities. You will collaborate with our cross-functional teams to ideate, develop, and own complex technical solutions on our fast-paced podcast and ad services technology platforms.

Our team at Spotify looks for candidates that share our common interest in distributed backend systems, their scalability, and continued development. In this role, you will closely collaborate with our internal customers to figure out the best API and backend system design to scalably meet their needs. You will build the backend systems that power our application, scale highly distributed systems, and continuously improve our engineering practices. Above all, this work will impact how the world experiences music.

What You'll Do

• Maintain and enhance Spotify’s reporting pipelines ensuring they remain business-critical.
• Build large-scale batch and real-time data pipelines for efficient collection, processing, and storage of measurement-related data.
• Make enhancements to data models and ETL code to improve pipeline efficiency and data quality.
• Work closely with stakeholders to understand business requirements and translate them into technical solutions.
• Write scalable, testable, maintainable, and high-quality code.
• Collaborate with engineers, product managers, and other stakeholders, embracing learning and leadership opportunities daily.
• Monitor daily execution, diagnose issues, and ensure SLAs are met by maintaining and fixing business-critical pipelines.
• Write distributed, high-volume services and know how to deploy and keep them running in production.

Who You Are

• Proven experience working with high-volume, heterogeneous data, ideally within distributed systems such as Hadoop, BigTable, Cassandra, GCP, AWS, or Azure
• Familiarity with data processing frameworks like Scio, GCP Dataflow, AWS EMR, and Apache Beam.
• Strong programming skills in Python, Java, or Scala.
• Commitment to agile software processes, data-driven development, reliability, and responsible experimentation.
• Hands-on experience with orchestration tools like Airflow or similar.
• Excellent problem-solving and communication skills.

Where You'll Be

• We offer you the flexibility to work where you work best! For this role, you can be within the North American region as long as we have a work location.
• This team operates within the EST time zone for collaboration.

The United States base range for this position is $160,091- $228,702 plus equity. The benefits available for this position include health insurance, six month paid parental leave, 401(k) retirement plan, monthly meal allowance, 23 paid days off, 13 paid flexible holidays, paid sick leave. These ranges may be modified in the future.

Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.

At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone. We have ways to request reasonable accommodations during the interview process and help assist in what you need. If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.

Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators. Everything we do is driven by our love for music and podcasting. Today, we are the world’s most popular audio streaming subscription service.