DATA Engineer - Azure data bricks - All GDS Locations

EY

Hyderabad, Pune District, Bengaluru

Hybrid

INR 1,200,000 - 2,000,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

EY in India seeks a strong Azure Data Engineer to design and implement scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL. The role requires expertise with Delta Lake, Spark SQL, DataFrame transformations, and CDC, plus experience with ADF, ADLS Gen2, and Medallion Architecture.

The ideal candidate will optimize Spark performance, ensure data quality, and contribute to dimensional modeling and SQL tuning.

Qualifications

  • Strong hands-on expertise in Azure Databricks, PySpark, Python and SQL.
  • Experience designing scalable ETL/ELT pipelines for large datasets.
  • Proficient with Delta Lake, Spark SQL, DataFrame transformations and CDC processing.
  • Experience with Azure Data Factory, ADLS Gen2 and Medallion Architecture.
  • Knowledge of Spark performance optimization and SQL query tuning.
  • Familiarity with Azure DevOps, Git, CI/CD, monitoring and production deployment.

Responsibilities

  • Design and develop scalable ETL/ELT pipelines for large datasets.
  • Optimize Spark jobs and ensure data quality.
  • Implement Medallion Architecture patterns with Delta Lake.
  • Collaborate using Azure Data Factory and ADLS Gen2.
  • Monitor pipelines and manage production deployments with CI/CD.

Skills

Azure Databricks
PySpark
Python
SQL
ETL/ELT pipelines
Delta Lake
Spark SQL
ADF
ADLS Gen2
Medallion Architecture
Spark optimization
Data quality
Dimensional modeling
SQL tuning
Azure DevOps
Git
CI/CD
Monitoring
Production deployment

Job description

a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; }

Job Summary

We are looking for a strong Azure Data Engineer with hands-on expertise in Azure Databricks, PySpark, Python, and SQL.
The candidate should have proven experience in designing and developing scalable ETL/ELT pipelines and processing large-volume datasets.
Strong knowledge of Delta Lake, Spark SQL, DataFrame transformations, joins, window functions, and incremental/CDC processing is required.
Experience with Azure Data Factory (ADF), ADLS Gen2, and Medallion Architecture is highly preferred.
The candidate should be proficient in Spark performance optimization, data quality, dimensional modeling, and SQL query optimization.
Hands-on exposure to Azure DevOps, Git, CI/CD, monitoring, and production deployment will be an added advantage.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Intact India • Gurugram District

On-site
INR 1,200,000 - 2,400,000
Data Engineer - Databricks
Data Engineer - Databricks

Onedatasoftware • Coimbatore District

On-site
INR 1,000,000 - 1,500,000
Data Engineering Consultant - Azure Databricks
Data Engineering Consultant - Azure Databricks

Optum • Hyderabad

On-site
INR 4,000,000 - 7,000,000
Azure Data Engineer
Azure Data Engineer

InfoBeans • Indore District, Pune District

On-site
INR 3,000,000 - 5,400,000
Azure Data Engineer (Databricks)
Azure Data Engineer (Databricks)

Aspyra Hr Services • Bengaluru, Delhi, Pune District

Hybrid
INR 800,000 - 1,200,000
Data Engineer-Information Technology
Data Engineer-Information Technology

Emcure Pharmaceuticals Limited • Pune District

On-site
INR 1,400,000 - 2,000,000
Databricks data engineer
Databricks data engineer

NYX TECH SOLUTION • Pune District

On-site
INR 1,200,000 - 2,100,000
Data Engineer-Azure Databricks
Data Engineer-Azure Databricks

Qloron • Mumbai

On-site
INR 1,200,000 - 1,800,000
Data Engineer
Data Engineer

OnPoint Insights • Mumbai Suburban

On-site
INR 1,200,000 - 1,800,000
Data Engineer
Data Engineer

Intact Insurance • India

On-site
INR 1,500,000 - 2,300,000