Principal Senior Data Engineer Databricks

Crescendo Global Leadership Hiring India; Hyderabad City Taluka; Hybrid; PKR 2,500,000 - 4,500,000; Full time

Job summary

Crescendo Global Leadership Hiring India is seeking a senior Data & Analytics Leader to guide a data engineering team and shape end-to-end data solutions. You will partner with business stakeholders, drive architecture decisions, and ensure robust AI-ready platforms.

The role emphasizes hands-on leadership, Azure-based data tech, and collaboration with data science and product teams to enable analytics-driven decision making.

Qualifications

• 10+ years of experience in Data & Analytics.
• Hands-on experience with Azure data engineering technologies.
• Strong programming and data processing skills in Python, SQL, and PySpark.
• Experience designing scalable data architectures and AI-ready platforms.

Responsibilities

• Partner with the business to understand problems and define data solutions with clear outcomes.
• Lead and mentor the data engineering team and drive engineering best practices.
• Provide technical leadership across architecture, development, troubleshooting and delivery.
• Collaborate with Data Architects, Data Scientists and Product Owners to enable AI-driven decisions.
• Drive innovation by challenging processes and proposing new solutions.
• Maintain and evolve the data platform for reporting and AI workloads.
• Define technical designs and execution plans with Data Product Owners.
• Design and maintain scalable data ingestion frameworks for batch and streaming use cases.

Skills

Python

SQL

PySpark

Azure data engineering

CI/CD

Tools

ADF Pipelines

Databricks Unity Catalog

DevOps

Job description

Responsibilities:
Partner closely with business to understand problems, shape requirements, and define data solutions with clear
outcomes.
Lead and mentor the data engineering team, driving engineering best practices and delivery excellence.
Provide technical leadership across architecture, development, troubleshooting, and delivery to drive execution speed
and quality.
Collaborate with Data Architects, Data Scientists, and Product Owners to enable business adoption of AI and advanced
analyticsdriven decision making.
Drive innovation and continuous improvement by challenging existing processes and proposing new solutions.
Maintain and evolve the data platform for business reporting and AI workloads.
Partner with Data Product Owners to define clear technical designs and execution plans aligned with business
outcomes.
Design and maintain scalable data ingestion frameworks supporting both batch and streaming use cases.

Experience & Skills:
10 years of experience in Data & Analytics, including hands-on experience with Azure data engineering technologies.
Strong hands-on experience with Python, SQL, and PySpark.
Strong understanding of Databricks Unity Catalog for data governance and security, along with CI/CD using DevOps.
Experience in designing scalable, secure data architectures and building AI-ready data platforms.
Proven ability to work with business and technical stakeholders to gather requirements and translate them into high-
impact data solutions.
Strong sense of ownership, accountability, and execution discipline across multiple initiatives.
Excellent communication skills, with the ability to handle high-impact business stakeholder interactions effectively.

EXPERTISE AND QUALIFICATIONS
Mandatory Skillset:
Recent 8+ years of experience in Azure services/tools

ADF Pipelines
Pyspark

ETL
SQL
PySpark.
Databricks Unity Catalog for data governance and security
Strong sense of ownership, accountability, and execution discipline across multiple initiatives.