Sr. Expert - Data Engineering

VOLKSWAGEN GROUP DIGITAL SOLUTIONS PRIVATE LIMITED; Pune District, Gurugram District, Bengaluru; On-site; INR 1,200,000 - 2,200,000; Full time

Job summary

VOLKSWAGEN GROUP DIGITAL SOLUTIONS PRIVATE LIMITED in Pune, India, is seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines in Azure.

You will work with Azure Databricks, ADLS, Synapse, and Data Factory, using Python (PySpark) and SQL/Postgres. The role focuses on data quality, governance, and cost-efficient cloud processing, with CI/CD for data workflows.

Qualifications

• Strong experience with Azure Data Services including Databricks, Python, and SQL/Postgres.

Responsibilities

• Design, build, and maintain scalable data pipelines using Azure data services and Azure Databricks.
• Develop and optimize ETL/ELT workflows for batch and real-time processing.
• Implement data solutions leveraging Azure Data Factory, ADLS, Synapse, and Databricks.
• Write SQL queries (SQL and Postgres) and Python (PySpark preferred).
• Collaborate with data analysts, data scientists, and stakeholders to understand data requirements.
• Ensure data quality, governance, and cost efficiency across platforms.
• Implement CI/CD pipelines and DevOps practices for data engineering workflows.
• Monitor and troubleshoot production data pipelines and workflows.
• Document data architecture, data flows, and technical designs.

Skills

Azure Databricks

Python

SQL/Postgres

ETL/ELT design

Cloud architecture

Tools

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

Databricks

Job description

Job Summary

Need an experienced Data Engineer with strong expertise in Azure Databricks, Azure Data Services, and Python to design, develop, and optimize scalable data pipelines and analytics solutions.

Responsibilities

• Design, build, and maintain scalable data pipelines using Azure data services and Azure Databricks
• Develop and optimize ETL/ELT workflows for batch and real-time processing
• Implement data solutions leveraging Azure Data Factory, Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Databricks
• Experience in writing SQL queries (SQL and Postgres DB)
• Write efficient, reusable, and scalable code in Python (PySpark preferred)
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements
• Ensure data quality, consistency, and governance across platforms
• Optimize data processing performance and cost efficiency in cloud environments
• Implement CI/CD pipelines and DevOps practices for data engineering workflows
• Monitor and troubleshoot production data pipelines and workflows
• Document data architecture, data flows, and technical designs

Requirements

• Strong experience in Azure Data Services, including: Azure Databricks, Python, SQL or Postgres
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Azure Synapse Analytics (or SQL Data Warehouse)