Principal Software Engineer - Ad Platform

RemoteWorker US; Santa Monica (CA); Remote; USD 184,300 - 247,100; Full time

Job summary

RemoteWorker US is hiring a Principal Software Engineer to lead the development of the Ad Platforms Data solutions. This role requires extensive experience with Databricks and advanced big data practices, including architecting scalable data solutions and optimizing data workflows. Ideal candidates have strong knowledge of data compliance and governance, and a passion for innovation in a team-driven environment. Compensation is competitive, aligning with the market standards for such highly skilled positions in California.

Qualifications

• 10+ years of experience in software engineering focused on big data platforms.
• Proven expertise in building scalable systems processing terabytes of data per hour.
• Hands-on experience with AWS cloud services and data solutions.

Responsibilities

• Architect and implement highly scalable ETL/ELT pipelines for both real-time and batch data processing.
• Design complex, multi-source data ecosystems with tools like Databricks, Delta Lake, and Snowflake.
• Optimize data pipelines leveraging cutting-edge tools like Airflow, Spark, and Flink.

Skills

Databricks

Spark

Flink

Airflow

Python

Java

Scala

Data Compliance

Data Governance

Education

Bachelor’s or Master's degree in Computer Science or related discipline

Job description

Principal Software Engineer - Ad Platform

Join to apply for the Principal Software Engineer - Ad Platform role at RemoteWorker US

Principal Software Engineer - Ad Platform

1 day ago Be among the first 25 applicants

Join to apply for the Principal Software Engineer - Ad Platform role at RemoteWorker US

On any given day at Disney Entertainment & ESPN Technology, we’re reimagining ways to create magical viewing experiences for the world’s most beloved stories while also transforming Disney’s media business for the future. Whether that’s evolving our streaming and digital products in new and immersive ways, powering worldwide advertising and distribution to maximize flexibility and efficiency, or delivering Disney’s unmatched entertainment and sports content, every day is a moment to make a difference to partners and to hundreds of millions of people around the world.

A few reasons why we think you’d love working for Disney Entertainment & ESPN Technology

• Building the future of Disney’s media business: DE&E Technologists are designing and building the infrastructure that will power Disney’s media, advertising, and distribution businesses for years to come.
• Reach & Scale: The products and platforms this group builds and operates delight millions of consumers every minute of every day – from Disney+ and Hulu, to ABC News and Entertainment, to ESPN and ESPN+, and much more.
• Innovation: We develop and execute groundbreaking products and techniques that shape industry norms and enhance how audiences experience sports, entertainment & news.
  
  

The products and platforms the Ad Platforms group builds and operates delight millions of consumers every minute of every day – from Disney+ and Hulu, to ABC News and Entertainment, to ESPN and ESPN+, and much more. We develop and execute groundbreaking products and techniques that shape industry norms and enhance how audiences experience sports, entertainment & news.

The Ad Platforms team is responsible for building and operating the technology that supports Disney’s industry-leading, multi-billion dollar global advertising business. Working across all of Disney’s streaming brands and linear channels, this team’s scope is wide ranging - they impact everything from core infrastructure development that powers Disney’s proprietary ad server, to reimagining viewer-first ad experiences and formats. This team’s award-winning, in-house built technologies are the backbone of Disney’s ad business and a critical element of the Company’s streaming future.

The Disney Ad Platforms Data team is at the forefront of building a modern advertising data platform to support our organization’s data, analytics and business intelligence needs. Our mission is to enable data engineers, data scientists, analysts, and decision-makers to derive value from our data ecosystem.

Job Summary

We are seeking a Principal Software Engineer with extensive experience in the Databricks platform and advanced big data engineering practices. This role involves architecting and implementing scalable, secure, and high-performing data solutions to power the Ad Platforms Data platform and support mission-critical data initiatives, including real-time and batch processing pipelines.

As a hands-on technical leader, you will design data architectures that scale to handle terabytes-per-hour throughput, ensuring optimal performance, compliance, and alignment with business needs. You will play a pivotal role in guiding teams, collaborating across departments, and driving innovation.

Responsibilities

Technical Leadership and Architecture

• Architect and implement highly scalable, secure, and efficient ETL/ELT pipelines for both real-time and batch data processing.
• Design complex, multi-source data ecosystems and ensure seamless integration with tools like Databricks, Delta Lake, and Snowflake.
• Optimize data pipelines and workflows for performance, cost, and reliability, leveraging cutting-edge tools like Airflow, Spark, Flink, and Databricks Delta Sharing.
• Lead the adoption of best practices in data warehousing, lakehouse architecture, and metadata management.
• Champion data security, privacy, and governance, implementing frameworks for lineage tracking, cataloging, and compliance (e.g., SOx, GDPR, CCPA).
• Drive implementation of Unity Catalog for data governance and cross-account querying.
  
  

Platform Development and Operational Excellence

• Able to assist in the set up and manage AWS environments, including S3, IAM Roles, Network Peering, and other infrastructure components.
• Oversee Airflow integration with Databricks Spark clusters, manage user access, and implement robust secret management solutions.
• Lead the CI/CD process with gated code releases, ensuring code quality and scalable deployment processes.
• Implement innovative solutions, from the exciting to the mundane; such as Databricks UniForm (Iceberg, Delta) format integration with Snowflake and basic tools to support cost projection, etc.
  
  

Data Engineering and Optimization

• Optimize Spark Streaming and ETL workflows, leveraging deep expertise in Spark performance tuning.
• Ensure best practices in data ingestion, transformation, and loading, focusing on highly performant and governed data models.
• Drive improvements in Delta Lake ecosystems and lakehouse architecture, including Delta Sharing for distributed querying.
  
  

Collaboration and Communication

• Collaborate closely with cross-functional teams, including Ad Server architects, to align on data strategies and designs.
• Foster open communication during architectural discussions, confidently defending decisions while welcoming feedback.
• Bridge the gap between diverse team requirements and technical capabilities to deliver cohesive, scalable solutions.
  
  

Innovation and Mentorship

• Stay ahead of emerging trends in big data technologies, cloud platforms (AWS, Azure, GCP), and distributed computing.
• Mentor engineers and architects, fostering a culture of innovation, knowledge sharing, and technical excellence.
  
  

Basic Qualifications

• Bachelor’s or Master's degree in Computer Science, Software Engineering or related technical discipline is highly desirable
• 10+ years of experience in software engineering with a focus on big data platforms.
• Proven expertise in building scalable systems capable of processing terabytes of data per hour.
• Hands-on experience with Databricks, Spark, Flink, Airflow, and Delta Lake ecosystems.
• Proficiency in Python (including PySpark), Java, or Scala.
• Extensive knowledge of data warehousing, data lakes, and real-time streaming systems.
• Hands-on experience with AWS cloud services and data solutions (e.g., S3, Redshift, Athena, IAM).
• Deep understanding of data compliance (e.g., SOX, GDPR) and governance.
• Strong skills in data lineage, cataloging, and metadata management.
  
  

Preferred Qualifications

• Experience in the media advertising domain, including linear and digital Ad Sales.
• Expertise with tools like ERStudio, ERWin, Alation, or Collibra for metadata management.
• Familiarity with Iceberg format integration and other modern data formats.
• Certifications in Databricks, AWS, or relevant big data technologies.
• Ability to manage/lead a team of high-performing data engineers and architects.
• Strong curiosity about how Disney delivers the Magic and a desire to be a part of it
  
  

#disneytech

The hiring range for this position in Los Angeles, CA is between $184,300 – $247,100 year and San Franscico, CA is between $201,900 - $270,700 and Seattle, WA is between $193,100 - $258,900. The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering and Information Technology

• Industries

  Staffing and Recruiting

Referrals increase your chances of interviewing at RemoteWorker US by 2x

Sign in to set job alerts for “Principal Software Engineer” roles.

Sr. Manager, Software Engineering - Accounts

Senior Manager / Director of Software Development

Santa Monica, CA $184,300.00-$270,700.00 2 months ago

Sr. Manager, Engineering, Advanced Technology & Innovation -Project Hire

Glendale, CA $138,900.00-$186,200.00 3 weeks ago

Principal Software Engineer (ML Focused) - League Studio, League Data Central

Los Angeles, CA $229,200.00-$319,500.00 4 weeks ago

Los Angeles, CA $160,000.00-$190,000.00 1 month ago

Sr. Manager, Software Engineering - Unpublished R&D Product

Software Engineering Manager, Members Experience

Los Angeles Metropolitan Area $197,895.00-$243,000.00 2 weeks ago

Los Angeles, CA $140,000.00-$160,000.00 1 month ago

Manager, Software Engineering - Teamfight Tactics, Core Tech

Sr Manager, Staff Software Engineering - Commercial Service Platform (REMOTE)

Santa Monica, CA $180,000.00-$250,000.00 1 week ago

Senior Software Engineer (Gameplay/Audio) - Unpublished R&D Product

Los Angeles, CA $140,000.00-$160,000.00 1 month ago

Senior Software Engineering Manager - REMOTE

Manager, Software Engineering, Messaging Infrastructure

Culver City, CA $89,300.00-$178,600.00 2 weeks ago

Manager, Software Engineering, Growth Notifications

Los Angeles, CA $195,000.00-$343,000.00 2 weeks ago

San Fernando, CA $130,000.00-$150,000.00 16 hours ago

Manager, Software Engineering - Content Access Platform

Sr. Manager, Propulsion Drives Embedded Software Engineering

Manager, Software Engineering - Internal Tools

Torrance, CA $200,100.00-$246,800.00 3 days ago

Agoura Hills, CA $95,000.00-$155,000.00 2 weeks ago

Senior Manager, Software Engineering, Camera & Lens

Culver City, CA $151,300.00-$261,500.00 2 weeks ago

Thousand Oaks, CA $140,000.00-$160,000.00 2 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.