Senior Data Engineer

Spotify; New York (NY); Remote; USD 160,000 - 229,000; Full time

Job summary

At Spotify, join us as a Senior Data Engineer where you will develop and enhance data pipelines that are crucial for business operations. Collaborate with cross-functional teams to ensure scalable API and backend solutions that impact how the world experiences music and podcasts. Enjoy a flexible working arrangement within the North American region and access a comprehensive benefits package that supports your well-being and personal life.

Benefits

Health insurance

Six month paid parental leave

401(k) retirement plan

Monthly meal allowance

23 paid days off

13 paid flexible holidays

Paid sick leave

Qualifications

• Proven experience in distributed systems (Hadoop, BigTable, etc.).
• Strong programming skills in Java or Scala required.
• Hands-on experience with orchestration tools like Airflow.

Responsibilities

• Build and maintain data pipelines for measurement-related data.
• Collaborate with stakeholders to translate business needs into solutions.
• Ensure SLAs are met by monitoring and diagnosing issues in pipelines.

Skills

High-volume data processing

Problem-solving

Communication

Tools

Hadoop

BigTable

Cassandra

GCP

AWS

Azure

Apache Beam

Job description

We are looking for a Senior Data Engineer to join our Advertising Product & Technology organization. Our teams are charged with scaling Spotify's Free Product and our Podcast Monetization Mission. You will work directly on an array of product features, advertiser publishing tools, and insights and reporting functionalities. You will collaborate with our cross-functional teams to ideate, develop, and own complex technical solutions on our fast-paced podcast and ad services technology platforms.

Our team at Spotify looks for candidates that share our common interest in distributed backend systems, their scalability, and continued development. In this role, you will closely collaborate with our internal customers to figure out the best API and backend system design to scalably meet their needs. You will build the backend systems that power our application, scale highly distributed systems, and continuously improve our engineering practices. Above all, this work will impact how the world experiences music.

What You'll Do

• Maintain and enhance Spotify’s reporting pipelines ensuring they remain business-critical.
• Build large-scale batch and real-time data pipelines for efficient collection, processing, and storage of measurement-related data.
• Work closely with stakeholders to understand business requirements and translate them into technical solutions.
• Write scalable, testable, maintainable, and high-quality code.
• Collaborate with engineers, product managers, and other stakeholders, embracing learning and leadership opportunities daily.
• Monitor daily execution, diagnose issues, and ensure SLAs are met by maintaining and fixing business-critical pipelines.
• Write distributed, high-volume services and know how to deploy and keep them running in production.

Who You Are

• Proven experience working with high-volume, heterogeneous data, ideally within distributed systems such as Hadoop, BigTable, Cassandra, GCP, AWS, or Azure
• Familiarity with data processing frameworks like Scio, GCP Dataflow, AWS EMR, and Apache Beam.
• Strong programming skills in Java or Scala.
• Commitment to agile software processes, data-driven development, reliability, and responsible experimentation.
• Hands-on experience with orchestration tools like Airflow or similar.
• Excellent problem-solving and communication skills.

Where You'll Be

• We offer you the flexibility to work where you work best! For this role, you can be within the North American region as long as we have a work location.
• This team operates within the EST time zone for collaboration.

The United States base range for this position is $160,091- $228,702 plus equity. The benefits available for this position include health insurance, six month paid parental leave, 401(k) retirement plan, monthly meal allowance, 23 paid days off, 13 paid flexible holidays, paid sick leave. These ranges may be modified in the future.

Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.

At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone. We have ways to request reasonable accommodations during the interview process and help assist in what you need. If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.

Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators. Everything we do is driven by our love for music and podcasting. Today, we are the world’s most popular audio streaming subscription service.