Consultant Data Engineer

Crescendo Global Leadership Hiring India; Hyderabad, Gurugram District, Bengaluru; Hybrid; INR 1,500,000 - 2,300,000; Full time

Job summary

Crescendo Global Leadership Hiring India is seeking a Consultant Data Engineer with strong Azure Databricks, PySpark and Spark SQL expertise to design scalable lakehouse solutions in Bengaluru, Hyderabad, or Gurugram.

You will build and orchestrate ETL pipelines using Azure Data Factory, ADLS and SQL, ensure data quality and reliability, monitor pipelines, and collaborate with cross‑functional teams to deliver enterprise analytics projects for global clients.

Qualifications

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Azure Databricks, Azure Data Factory (ADF), and ADLS.
• Proficiency in PySpark, Spark SQL, ETL, and advanced SQL.
• Experience building scalable data pipelines and enterprise data platforms.
• Excellent analytical, problem-solving, and stakeholder management skills.

Responsibilities

• Design and develop scalable data pipelines using PySpark and Spark SQL on Azure Databricks.
• Build and orchestrate ETL workflows using Azure Data Factory (ADF) and ADLS.
• Develop and maintain modern Lakehouse architecture while ensuring data quality and reliability.
• Monitor production data pipelines, troubleshoot issues, and perform root cause analysis.
• Collaborate with cross-functional teams to deliver scalable data solutions and analytics initiatives.

Skills

Azure Databricks

PySpark

Spark SQL

ETL

Azure Data Factory

ADLS

SQL

Data Engineering

Tools

ADF

ADLS

Job description

Consultant Data Engineer Databricks | Azure | PySpark | Spark SQL - Bengaluru / Hyderabad / Gurugram

Looking for an experienced Data Engineer with strong expertise in Azure Databricks, PySpark, Spark SQL, Azure Data Factory (ADF), ADLS, ETL, and SQL. You will design scalable data pipelines, build modern Lakehouse solutions, and enable enterprise analytics in a collaborative, fast-paced environment.

Your Future Employer –

A leading global AI and analytics consulting organization delivering cutting-edge data engineering, analytics, and AI solutions for enterprise clients worldwide.

Location –

Bengaluru / Hyderabad / Gurugram

Responsibilities –

1. Design and develop scalable data pipelines using PySpark and Spark SQL on Azure Databricks.
2. Build and orchestrate ETL workflows using Azure Data Factory (ADF) and ADLS.
3. Develop and maintain modern Lakehouse architecture while ensuring data quality and reliability.
4. Monitor production data pipelines, troubleshoot issues, and perform root cause analysis.
5. Collaborate with cross-functional teams to deliver scalable data solutions and analytics initiatives.

Requirements –

1. 5+ years of experience in Data Engineering.
2. Strong hands-on experience with Azure Databricks, Azure Data Factory (ADF), and ADLS.
3. Proficiency in PySpark, Spark SQL, ETL, and advanced SQL.
4. Experience building scalable data pipelines and enterprise data platforms.
5. Excellent analytical, problem-solving, and stakeholder management skills.

What is in it for you –

1. Opportunity to work on large-scale cloud data engineering projects.
2. Exposure to modern Azure and Databricks technologies.
3. Collaborative and innovation-driven work culture.
4. Excellent learning and career growth opportunities.
5. Competitive compensation and benefits.

Crescendo Global does not discriminate on the basis of race, religion, color, origin, gender, sexual orientation, age, marital status, veteran status, or disability status.