Data Engineer, Personalization

Spotify; New York (NY); Remote; USD 160,000 - 229,000; Full time

Job summary

Spotify is seeking a Data Engineer, Personalization to join their team in New York. This role involves building scalable data pipelines and collaborating with cross-functional teams to enhance podcast and audiobook experiences. Candidates should have a passion for data and experience with data processing frameworks. The position offers a competitive salary and benefits, highlighting Spotify's commitment to inclusivity and innovation.

Benefits

Health insurance

Paid parental leave

401(k) retirement plan

Monthly meal allowance

Paid time off

Flexible holidays

Paid sick leave

Qualifications

• Experience building high-scale production data pipelines.
• Proficient in data modeling, data access, and storage optimization.
• Curiosity about ML data processing and inference pipelines.

Responsibilities

• Develop new data pipelines and maintain existing systems.
• Collaborate with ML engineers to develop training datasets.
• Create monitoring tools for AI product quality.

Skills

Data modeling

Data access optimization

Data storage optimization

Collaboration

Machine Learning processing

Tools

Scio

Apache Beam

Ray

PyTorch

Google Cloud Platform

Job description

Join to apply for the Data Engineer, Personalization role at Spotify

Join to apply for the Data Engineer, Personalization role at Spotify

Get AI-powered advice on this job and more exclusive features.

Personalization’s Hulk squad produces Human Understandable Language Knowledge to enrich content understanding. We utilize Large Language Models to understand podcasts and audiobooks, building reliable, scalable systems to distribute that knowledge to Spotify internal teams, users, and creators. We are looking for a Data Engineer with interest in ML techniques to join our team and help build the future of podcast and audiobook listening experiences for millions of listeners at Spotify. This is a unique opportunity to help develop and shape Spotify recommendations. You’ll grow your skills in engineering at scale, work with a cross-functional team of Machine Learning Engineers, Backend Engineers, Data Engineers, and researchers, and join a super motivated and supportive team.

What You'll Do

• Work with large-scale data pipelines using data processing frameworks like Scio (built on Apache Beam), Hendrix ML, Ray, PyTorch, and Dataflow and other Google Cloud Platform offerings
• Support streaming and batch inference of core LLM models
• Develop new data pipelines to meet emerging needs and maintain our existing data systems as necessary to ensure they are reliable, scalable, and well tested
• Work with ML engineers to develop training and evaluation datasets for Generative LLM models
• Own the development of data solutions from architecture to delivery
• Collaborate with other backend and data engineers, ML engineers, data scientists, researchers, and other stakeholders
• Develop monitoring tools to automatically track AI product quality. This likely involves the use of LLMs to evaluate results as well as closer monitoring of performance indicators, and creation and maintenance of metrics datasets and dashboards
• Work in an agile team to continuously experiment, iterate and deliver on new product objectives
• Share knowledge, promote standard methodologies, making your team the best version of itself through mentorship and constructive accountability
  
  

Who You Are

• You are passionate about data and proficient in data modeling, data access, and data storage optimization
• You have experience building high-scale production data pipelines with one or more data processing frameworks such as Scio (built on Apache Beam), and major props if you know Ray
• You might have worked with orchestration frameworks such as Flyte
• You care about agile software processes, data-driven development, reliability, and responsible experimentation
• You value collaboration and partnership within teams
• Experience, exposure, or curiosity about ML Data processing and inference pipelines, LLM fine-tuning, and/or using LLMs for data analysis
• Bonus if you have experience with NRT (near real time) systems
• Bonus if you experience with prompt engineering for LLMs and prompt optimization tools like DSPy
  
  

Where You'll Be

• We offer you the flexibility to work where you work best! For this role, you can be within the North America region as long as we have a work location.
• This team operates within the Eastern Standard time zone for collaboration.
  
  

The United States base range for this position is $160,091- $228,702 plus equity. The benefits available for this position include health insurance, six month paid parental leave, 401(k) retirement plan, monthly meal allowance, 23 paid days off, 13 paid flexible holidays, paid sick leave. These ranges may be modified in the future.

Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.

At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone. We have ways to request reasonable accommodations during the interview process and help assist in what you need. If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.

Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators. Everything we do is driven by our love for music and podcasting. Today, we are the world’s most popular audio streaming subscription service.

Seniority level

• Seniority level

  Entry level

Employment type

• Employment type

  Full-time

Job function

Referrals increase your chances of interviewing at Spotify by 2x

Get notified about new Data Engineer jobs in New York, NY.

New York, NY $46.63-$134,000.00 1 week ago

New York City Metropolitan Area $119,000.00-$137,000.00 1 week ago

New York, NY $114,000.00-$171,000.00 2 weeks ago

New York, NY $175,000.00-$230,000.00 1 week ago

Data Engineer, Analytics (Technical Leadership)

New York, NY $206,000.00-$281,000.00 2 weeks ago

New York, NY $130,000.00-$200,000.00 2 weeks ago

New York, NY $170,000.00-$250,000.00 1 week ago

New York, NY $170,000.00-$240,000.00 3 months ago

New York, NY $173,000.00-$242,000.00 2 weeks ago

New York, NY $145,000.00-$204,000.00 2 weeks ago

Data Engineer III, Go-to-Market Operations

New York, NY $120,000.00-$170,000.00 1 month ago

New York, NY $56.25-$173,000.00 2 weeks ago

New York, NY $90,000.00-$130,000.00 3 months ago

New York, NY $24,000.00-$72,000.00 5 days ago

New York, NY $150,000.00-$200,000.00 1 month ago

Software Engineer, Search Infrastructure

New York, NY $220,000.00-$260,000.00 6 days ago

Member of Technical Staff - Data Engineer

New York, NY $117,200.00-$294,000.00 1 week ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.