Principal, Data Engineer

Walmart Global Tech India; Chennai District; On-site; INR 1,800,000 - 2,500,000; Full time

Job summary

Walmart Global Tech India in Chennai is seeking a senior data engineer to transform and elevate its core data platform using Agentic AI. Ideal candidates will have over 12 years of experience in computer science, with extensive knowledge of Big Data technologies and cloud-native systems. Responsibilities include leading the modernization of legacy systems, developing scalable data platforms, and mentoring engineers. The position offers competitive compensation and growth opportunities in a dynamic environment.

Qualifications

• Bachelor’s degree in computer science or related discipline with 12+ years’ experience.
• Minimum 10 years of experience in Big Data and distributed computing.
• Strong experience with cloud-native ecosystems (GCP), including BigQuery.
• Expertise in batch and streaming technologies (Kafka, Spark).
• Fluency in Python, Java, or Scala.

Responsibilities

• Transform Walmart’s core data platform using Agentic AI for Global Marketplace applications.
• Develop scalable data platforms for billions of events per day.
• Lead modernization of legacy data lakes into composable platforms.
• Mentor Staff and Senior Engineers across Walmart Global Tech AI and Data team.
• Design, build, test, and deploy solutions for a multi-billion-dollar business.

Skills

Cloud-native ecosystems (GCP)

Big Data technologies (Hadoop, Spark, Hive)

Programming in Java, Scala, Python

Semantic modeling

Agentic & AI‑Ready Systems Knowledge

Data pipeline construction

Education

Bachelor's degree in computer science or related discipline

Tools

BigQuery

Kafka

Spark Structured Streaming

Job description

Responsibilities

• Transform and evolve Walmart’s core data platform using Agentic AI across batch, streaming, and hybrid systems to support data-powered Global Marketplace applications, analytics, operational intelligence, and AI-native workloads.
• Develop data platforms that scale to billions of events per day, supporting near real‑time retail decisions and deep analytical insights.
• Develop agent‑ready data, ensuring data products are discoverable, semantically rich, and optimized for LLMs, copilots, and multi‑agent workflows.
• Influence multiple teams and organizations through technical leadership and clear architectural direction—without direct authority.
• Develop fault‑tolerant, resilient systems that support mission‑critical retail and marketplace operations.
• Lead modernization of legacy data lakes and pipelines into composable, event‑driven, and agent‑aware platforms.
• Enable incremental modernization that delivers immediate value while reducing long‑term complexity and operational risk.
• Mentor Staff and Senior Engineers across Walmart Global Tech AI and Data team.
• Own and build scalable and highly available distributed systems in production.
• Develop APIs and services for enabling data for dependent systems and applications.
• Implement best‑in‑class data pipelines to ensure on‑time availability of data and insights.
• Identify data sources that are fit for purpose.
• Create conceptual, physical, and logical data models by analyzing complex data elements and relationships.
• Enable data scientists, business and product partners to fully leverage the platform.
• Transform business requirements into code, analytical reports, and tools.
• Design, build, test, and deploy cutting‑edge solutions at scale, impacting a multi‑billion‑dollar business.
• Adopt and build AI Agents for discovering data assets.
• Collaborate with product owners and technical leads on project delivery.
• Research, learn, and support both new projects and production workloads.
• Provide business insights using internal tools, databases, and industry data.
• Own a data subject and ensure its availability and accuracy.
• Develop long‑term data platform architecture in partnership with business units and engineering teams.
• Build data tools for analytics and data‑scientist teams to optimize within the Walmart ecosystem.
• Create training documentation and train end users on data modeling.
• Document requirements, data lineage, and subject matter in business and technical terms.
• Recommend ways to improve data reliability, efficiency, and quality.

Qualifications

• Bachelor’s degree in computer science or related discipline with 12+ years’ experience.
• Minimum 10 years of experience in Big Data and distributed computing.
• Minimum 10 years of experience programming in Java, Scala, Python, SpringBoot & Nodejs.
• Strong experience with cloud‑native ecosystems (GCP), including BigQuery, Serverless, Pub/Sub, or equivalent.
• Expertise in batch and streaming technologies (Kafka, Spark Structured Streaming, Flink, Druid, etc.).
• Experience working with hybrid architectures that support both real‑time operations and analytical workloads.
• Agentic & AI‑Ready Systems Knowledge.
• Strong understanding of semantic modeling, embeddings, knowledge graphs, and vector indexing.
• Experience supporting RAG, context‑aware AI, and agent orchestration through data‑platform design.
• Ability to reason about schema design, latency, storage formats, and their impact on AI behavior and outcomes.
• Fluency in Python, Java, or Scala.
• Deep experience with Spark/PySpark and large‑scale SQL optimization.
• Strong systems thinking, performance tuning, and operational excellence mindset.
• Demonstrated ability to lead through influence in complex, matrixed organizations.
• Executive‑level communication skills, connecting technical strategy to business and customer impact.
• Proven experience building pipelines on Big Data Technologies/Stack—Hadoop, Spark, Hive, Presto, Kafka, Airflow Scheduler and GCP suite of data tools.
• Strong knowledge of deploying and managing applications in GCP.
• Strong scripting skills for processing large amounts of data and high proficiency in SQL.
• Solid knowledge of Linux systems and troubleshooting in complex, distributed, multi‑tier architectures.
• Experience with secure, scalable, and highly available services.
• Experience with data science and machine learning is a plus.
• Excellent hands‑on working knowledge of object‑oriented/functional scripting languages: Python, Java, C++, Scala, and micro‑service development.