Lead Azure Data Engineer _ Exp: 8+ Years

Atyeti Inc; Pune District; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Atyeti Inc. is hiring an experienced Senior Data Engineer in Pune to design, build, and optimize enterprise-scale Azure data platforms. You will work on ingestion, transformation, and orchestration pipelines using Databricks, PySpark, and ADF.

The role emphasizes data modeling, Delta Lake layers, and collaboration with stakeholders to deliver scalable and reliable data solutions. Strong communication and Agile experience are required.

Qualifications

• 7+ years of experience in Data Engineering and Data Warehousing.
• 4+ years of hands-on experience with Azure Databricks and PySpark.
• Experience in Agile/Scrum environments.
• Excellent communication and problem-solving skills.
• Bachelor's degree in Computer Science, Information Technology, or a related field.

Responsibilities

• Design, build, and maintain scalable data ingestion, transformation, and delivery pipelines.
• Develop ETL/ELT solutions using Azure Databricks, PySpark, and ADF.
• Develop and optimize Databricks notebooks, workflows, and scheduled jobs.
• Implement Delta Lake-based Bronze, Silver, and Gold data layers.
• Integrate data from databases, APIs, files, and cloud storage platforms.
• Ensure data quality, performance, reliability, and operational support for production pipelines.
• Collaborate with architects, analysts, and business stakeholders to deliver data solutions.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Azure Data Factory

Databricks Jobs & Workflows

Azure Data Lake

ADLS Gen2

Azure SQL

ETL/ELT Design

Data Ingestion

Data Transformation

Data Modeling

Data Warehousing

Performance Tuning

CI/CD

Git

Data Quality

Monitoring

Production Support

Education

Bachelor's degree in CS/IT

Tools

Azure Databricks

Azure Data Factory

Delta Lake

PySpark

ADLS Gen2

Job description

Job Overview :

We are seeking an experienced Senior Data Engineer to design, build, optimize, and support enterprise-scale data platforms on Azure. The ideal candidate will possess strong expertise in data ingestion, transformation, orchestration, and data warehousing using technologies such as Azure Databricks, Azure Data Factory (ADF), Delta Lake, PySpark, and Azure SQL. The candidate will work closely with business stakeholders, data architects, analysts, and application development teams to deliver scalable, high-performance, and reliable data solutions.

Responsibilities:

• Design, develop, and maintain scalable data ingestion, transformation, and delivery pipelines.
• Build ETL/ELT solutions using Azure Databricks, PySpark, and Azure Data Factory (ADF).
• Develop and optimize Databricks notebooks, workflows, and scheduled jobs.
• Implement and maintain Delta Lake-based Bronze, Silver, and Gold data layers.
• Integrate data from databases, APIs, files, and cloud storage platforms.
• Ensure data quality, performance, reliability, and operational support for production pipelines.
• Collaborate with architects, analysts, and business stakeholders to deliver data solutions.
• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7+ years of experience in Data Engineering and Data Warehousing.
• 4+ years of hands-on experience with Azure Databricks and PySpark.
• Experience working in Agile/Scrum environments.
• Excellent communication and problem-solving skills.

Skills:

Azure Databricks (PySpark, Spark SQL, Delta Lake), Azure Data Factory (ADF) ,Databricks Jobs & Workflows, Azure Data Lake, Storage (ADLS Gen2), Azure SQL Database /SQL Server, ETL/ELT Design & Development, Data Ingestion &Data Transformation, Data Modeling &Data Warehousing, Performance Tuning & Optimization, CI/CD, Git, Data Quality, Monitoring, and Production Support