Data Engineer 6+ Exp (Azure, ADF, Databricks, PySpark)

Innova ESI; Bengaluru; On-site; USD 908 - 1,363; Full time

Job summary

Join a forward-thinking company as a Talent Acquisition Specialist II focused on IT hiring. This exciting role requires over 6 years of experience in Data Engineering, where you'll develop and optimize data pipelines using cutting-edge tools like Azure Data Factory and Databricks. Collaborate with cross-functional teams to drive innovative data-driven solutions and implement robust data governance practices. If you're passionate about pushing the boundaries of data engineering and thrive in a collaborative environment, this opportunity is perfect for you!

Qualifications

• 6+ years of experience in Data Engineering with a focus on data pipelines.
• Proficiency in PySpark and SQL for data optimization.

Responsibilities

• Develop and optimize data pipelines using Azure Data Factory and Databricks.
• Implement SCD2 in Delta Tables and leverage Unity Catalog for data governance.

Skills

Data Engineering

PySpark

SQL

Data Pipeline Optimization

Data Governance

CI/CD Practices

Azure Synapse Analytics

Data Lakes

Power BI

Kafka/Event Hub

Education

Azure Certifications (DP-203, DP-900)

Databricks Certifications

Tools

Azure Data Factory (ADF)

Azure Databricks

Unity Catalog

Job description

1 day ago Be among the first 25 applicants

Direct message the job poster from Innova ESI

Working as Talent Acquisition Specialist II IT Hiring II IT Recruitment

Experience: 6+ years

What You'll Do:

1. Develop and optimize data pipelines using Azure Data Factory (ADF) & Azure Databricks
2. Implement Slowly Changing Dimension Type 2 (SCD2) in Delta Tables
3. Leverage Unity Catalog for robust data governance
4. Collaborate with cross-functional teams to drive data-driven solutions

Qualifications:

• 6+ years of Data Engineering experience
• Proficient in PySpark, SQL, and data pipeline optimization
• Knowledge of data governance, security, and CI/CD practices
• Experience with Azure Synapse Analytics, Data Lakes, Power BI
• Knowledge of Kafka/Event Hub for real-time data ingestion
• Azure (DP-203, DP-900) or Databricks certifications

Be part of a team that values innovation, collaboration, and continuous growth. Work on exciting projects that push the boundaries of data engineering!

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Information Technology

Industries

IT Services and IT Consulting