Principal Data Engineer

Jobtailor; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Walmart is seeking an Architect to design scalable data systems handling billions of ad and customer signals. The role emphasizes low latency, high throughput, resilience, and cost-efficient scaling across real-time and batch workloads.

You will unify ad data across campaigns, identity, audiences, measurement, and CRM, building multi-tenant platforms with governance and observability. Strong leadership across the advertising lifecycle is essential.

Qualifications

• Bachelor’s degree in Computer Science.
• 5 years' experience in software engineering or related field; or 7 years’ experience in software engineering or related field.
• 4 years' experience in data engineering, database engineering, business intelligence, or business analytics.
• Deep expertise designing billion-scale, real-time and near-real-time distributed data systems.
• Strong AdTech/MarTech expertise across retail media, bidding, targeting, identity, attribution, measurement, reporting, privacy-safe collaboration.

Responsibilities

• Architect scalable data systems processing billions of advertising and customer signals across real-time, near-real-time, micro-batch, and batch workloads.
• Ensure low latency, high throughput, resilience, replayability, idempotency, and cost-efficient scaling.
• Unify advertising data across campaigns, identity, audiences, measurement, optimization, CRM, and advertiser lifecycle.
• Build reusable, configuration-driven, multi-tenant platforms with standardized schemas, contracts, governance, observability, privacy, and deployment patterns.
• Modernize legacy pipelines for real-time optimization, reporting, activation, and machine learning.
• Lead agentic data enablement for discovery, pipeline automation, monitoring, diagnostics, cost optimization, onboarding, recommendations, and secure, explainable production guardrails.
• Define workload patterns across streaming, near-real-time, micro-batch, and batch architectures.
• Provide technical leadership across the full advertising lifecycle.
• Establish governed agentic data capabilities and AI-ready products with security, reliability, explainability, safety, and measurable outcomes.
• Collaborate across Walmart U.S., Sam’s Club, and international advertising businesses

Skills

Data Engineering

Real-Time Processing

GCP Cloud Platforms

Spark and Kafka

AdTech/MarTech Expertise

Education

Bachelor's Degree in Computer Science

Master's Degree in Computer Science

Tools

Spark

Flink

Kafka

GCP

Dataproc

Pub-Sub

Airflow

Job description

• Architect scalable data systems processing billions of advertising and customer signals across real-time, near-real-time, micro-batch, and batch workloads


• Ensure low latency, high throughput, resilience, replayability, idempotency, and cost-efficient scaling


• Unify advertising data across campaigns, identity, audiences, measurement, optimization, CRM, and advertiser lifecycle


• Build reusable, configuration-driven, multi-tenant platforms with standardized schemas, contracts, governance, observability, privacy, and deployment patterns


• Modernize legacy pipelines for real-time optimization, reporting, activation, and machine learning


• Lead agentic data enablement for discovery, pipeline automation, monitoring, diagnostics, cost optimization, onboarding, recommendations, and secure, explainable production guardrails


• Define workload patterns across streaming, near-real-time, micro-batch, and batch architectures


• Provide technical leadership across the full advertising lifecycle


• Establish governed agentic data capabilities and AI-ready products with security, reliability, explainability, safety, and measurable outcomes


• Collaborate across Walmart U.S., Sam’s Club, and international advertising businesses

Requirements

• Bachelor’s degree in Computer Science and 5 years' experience in software engineering or related field; or 7 years’ experience in software engineering or related field; or Master's degree in Computer Science and 3 years' experience in software engineering or related field


• 4 years' experience in data engineering, database engineering, business intelligence, or business analytics


• Deep expertise designing billion-scale, real-time and near-real-time distributed data systems


• Experience with Spark, Flink, Kafka, streaming, orchestration, and compute/storage optimization


• Strong experience with GCP or equivalent cloud platforms


• Experience with BigQuery, Dataproc, Dataflow, Pub-Sub, Airflow, and modern data infrastructure


• Deep understanding of low-latency architectures, partitioning, backpressure, replay, consistency, fault tolerance, resilience, recovery, and cost efficiency


• Proven Principal-level leadership shaping architecture, standards, strategic trade-offs, design reviews, and senior technical talent


• Strong AdTech/MarTech expertise across retail media, bidding, targeting, identity, attribution, measurement, reporting, CTV, and privacy-safe collaboration is desirable


• Preferred experience with ETL tools and large data sets in the cloud


• Preferred Master's degree in Computer Science or related field and 5 years' experience in software engineering or related field


• Preferred knowledge of WCAG 2.2 AA standards, assistive technologies, and digital accessibility best practices

Demonstrates expertise in architecting scalable data systems for real-time and batch processing, with a focus on low-latency, high throughput, and resilience. Proven ability to lead technical initiatives across advertising data lifecycle while ensuring security, explainability, and cost efficiency.

Highest-signal resume keywords

• Data Engineering


• Real-Time Distributed Data Systems


• GCP Cloud Platforms


• Spark and Kafka


• AdTech/MarTech Expertise

Hard Skills

• Data Engineering


• Database Engineering


• Business Intelligence


• Real-Time Processing


• Distributed Systems Design


• ETL Tools


• BigQuery


• Dataflow


• Streaming Architectures


• Compute/Storage Optimization

Soft Skills

• Technical Leadership


• Collaboration


• Strategic Trade-Offs


• Design Reviews

Certifications & Qualifications

• Bachelor’s Degree in Computer Science


• Master's Degree in Computer Science

Industry Keywords

• Advertising Data


• Retail Media


• Bidding


• Targeting


• Attribution


• Measurement


• Privacy-Safe Collaboration


• WCAG 2.2 AA Standards

Tools & Technologies

• Spark


• Flink


• Kafka


• GCP


• Dataproc


• Pub-Sub


• Airflow