Data and AI Engineer

Western Digital; Bengaluru; On-site; INR 2,500,000 - 5,000,000; Full time

Job summary

Western Digital in Bengaluru, India seeks a data/ML engineer to design scalable ETL/ELT pipelines ingesting data from SAP S/4HANA, Oracle, Salesforce and APIs into enterprise data lakes.

You will develop data products on Databricks, enforce data quality and governance with Purview/Alation/Unity Catalog, and build ML models using Python and PySpark.

The role includes GenAI/LLM solutions, RAG pipelines, vector databases, and MLOps for CI/CD and retraining across cloud platforms.

Qualifications

• 4-6 years of experience in data engineering, ML engineering, or applied AI roles.
• Proven track record of delivering 2-3 production-grade data or AI projects end-to-end.
• Experience working in global team environments is a plus.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines ingesting data from SAP S/4HANA, Oracle, Salesforce and APIs.
• Develop and optimize data products (medallion architecture) on Databricks.
• Implement data quality, lineage, and governance controls with Purview/Alation/Unity Catalog.
• Build, deploy, and monitor ML models using Python, PySpark, scikit-learn, TensorFlow, or PyTorch.
• Develop GenAI and LLM-powered solutions including RAG pipelines, embeddings, vector databases, and prompt engineering.
• Operationalize models with MLOps practices and CI/CD for ML, versioning, drift monitoring, and retraining.
• Deploy data/AI workloads on Azure, AWS, or GCP using services like ADF, Databricks, Glue, Lambda, and Kubernetes.

Skills

ETL/ELT pipelines

Python

PySpark

GenAI concepts

ML model development

Tools

Databricks

Purview

Alation

Unity Catalog

Azure

AWS

GCP

Job description

Job Description

• Design, build, and maintain scalable ETL/ELT pipelines to ingest structured and unstructured data from SAP S/4HANA, Oracle, Salesforce, and third-party APIs into enterprise data lakes/lakehouses.
• Develop and optimize data products (medallion architecture) on platforms like Databricks
• Implement data quality, lineage, and governance controls using tools such as Purview, Alation, or Unity Catalog.
• Build, deploy, and monitor ML models (classification, regression, forecasting, anomaly detection) using Python, PySpark, scikit-learn, TensorFlow, or PyTorch.
• Develop GenAI and LLM-powered solutions including RAG (Retrieval-Augmented Generation) pipelines, embeddings, vector databases (Pinecone, Azure AI Search), and prompt engineering for Copilot/Azure OpenAI/Bedrock.
• Operationalize models through MLOps practices CI/CD for ML, model versioning, drift monitoring, and automated retraining (MLflow, Azure ML, SageMaker, Vertex AI)
• Deploy and manage data/AI workloads on Azure, AWS, or GCP, leveraging services like ADF, Databricks, Glue, Lambda, Functions, and Kubernetes.

Qualifications

• 4 6 years of professional experience in data engineering, ML engineering, or applied AI roles.
• Proven track record of delivering at least 2 3 production-grade data or AI projects end-to-end
• Working in global team ecosystem is a plus

Preferred Qualifications

• Experience with enterprise ERP data (SAP S/4HANA, Oracle) and integration patterns (CDC, IDoc, OData, APIs).
• Exposure to supply chain, procurement, Logistics analytics use cases (spend, demand forecasting, supplier risk).
• Familiarity with Copilot Studio, or Agentic AI frameworks.