Senior Data Engineer: Scalable Pipelines, Spark & Cloud AI

Walmart; Bentonville (AR); On-site; USD 90,000 - 180,000; Full time

Benefits offered by this job

Health benefits

401(k)

Stock purchase

PTO & leave

Live Better U education

Job summary

Walmart is seeking a Data Engineer III to design, develop, and maintain scalable data pipelines and architectures that empower analytics and decision-making. You will collaborate across domains to translate complex requirements into effective data solutions while advancing data governance and enterprise data strategies.

The External Data Engineering team focuses on integrating large external datasets, applying advanced modeling, and using Spark and cloud platforms to deliver high-quality,

Qualifications

• Bachelor’s degree in Computer Science and 2 years' software engineering or related field.
• Option 2: 4 years’ software engineering or related field.
• Option 3: Master’s degree in Computer Science.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes to support business analytics and reporting needs.
• Collaborate with cross-functional teams to identify data requirements and translate them into data models and solutions.
• Implement data integration and ingestion strategies using cloud and big data technologies such as Spark and data lakes.
• Ensure data quality, governance, and compliance with regulatory standards throughout the data lifecycle.
• Optimize data storage, transformation, and retrieval processes to enhance performance and scalability.
• Develop and test code for data processing, supporting deployment and documentation efforts.
• Stay current with emerging data engineering trends and best practices to drive continuous improvement.

Skills

Scala & Spark

BigQuery

GCS

Dataproc

Kafka

Spark Structured Streaming

SQL

NoSQL

Git

CI/CD

Airflow

Tableau

Looker

GenAI

Education

Bachelor’s degree in Computer Science

Master’s degree in Computer Science

Tools

Git

CI/CD pipelines

Airflow

Job description

Walmart is seeking a Data Engineer III to design, develop, and maintain scalable data pipelines and architectures that empower analytics and decision-making. You will collaborate across domains to translate complex requirements into effective data solutions while advancing data governance and enterprise data strategies.

The External Data Engineering team focuses on integrating large external datasets, applying advanced modeling, and using Spark and cloud platforms to deliver high-quality,