Senior Data Engineer - Databricks

Anblicks; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Anblicks is seeking a Senior Data Engineer with a strong foundation in data warehousing and cloud-native data engineering. You will lead the design and development of modern data pipelines, driving enterprise data transformation initiatives across Databricks, Azure, and Fabric.

This strategic role requires hands-on experience with Databricks, PySpark, and Delta Lake, and a proven ability to mentor engineers while aligning data solutions with business goals.

Qualifications

• 6+ years of experience in Data Engineering, with focus on data warehousing, distributed processing, and cloud platforms.
• 4+ years of hands-on experience in Databricks: PySpark, Delta Lake.
• Proficient in SQL, data modeling, and performance tuning for analytical workloads.

Responsibilities

• Design and implement scalable, high-performance data pipelines on Databricks using PySpark, SQL, and Delta Lake.
• Develop and optimize data lakehouse architectures, ensuring robust data ingestion, transformation, and storage.
• Build and manage ETL/ELT workflows integrating structured and semi-structured data from diverse sources.
• Apply best practices in data modeling, partitioning, and performance tuning.
• Leverage Databricks on Azure for large-scale distributed processing, including job orchestration and notebook management.
• Collaborate with platform teams to enable secure, scalable, and governed data environments across cloud ecosystems.
• Partner with business analysts, architects, and client stakeholders to translate business needs into scalable data solutions.
• Ensure data quality, lineage, and observability through tools like Unity Catalog, Great Expectations, or Azure Purview.
• Guide and mentor junior engineers, enforce code and design standards, and drive adoption of reusability and best practices.
• Document technical designs, data flow diagrams, and data governance procedures.

Skills

Data engineering

SQL & data modeling

Leadership & mentoring

Tools

Databricks

Azure Synapse/ADF

Delta Lake

Great Expectations

Job description

Experience: 6+ Years

Immediate Joiners Preferred (Max Joining by 1st Oct)

Shortlisted Candidates - Face to Face Interview on 19th Sept (This Saturday) Only.

About the Role

We are looking for a Senior Data Engineer with a strong foundation in data warehousing, cloud-native data engineering, and deep hands-on experience with Databricks. Exposure to Data Modelling is a strong plus. In this role, you will lead the design and development of modern data pipelines, drive enterprise data transformation initiatives, and contribute to a future-ready data architecture that blends best practices across Databricks, Azure, and Fabric.

This is a strategic role for a data expert who enjoys solving high-impact business problems through modern data platforms, scalable pipelines, and cloud-native tools.

Key Responsibilities

• Design and implement scalable, high-performance data pipelines on Databricks using PySpark, SQL, and Delta Lake.
• Develop and optimize data lakehouse architectures, ensuring robust data ingestion, transformation, and storage.
• Build and manage ETL/ELT workflows integrating structured and semi-structured data from diverse sources.
• Apply best practices in data modeling, partitioning, and performance tuning.
• Leverage Databricks on Azure for large-scale distributed processing, including job orchestration and notebook management.
• Collaborate with platform teams to enable secure, scalable, and governed data environments across cloud ecosystems.
• Partner with business analysts, architects, and client stakeholders to translate business needs into scalable data solutions.
• Ensure data quality, lineage, and observability through tools like Unity Catalog, Great Expectations, or Azure Purview.
• Guide and mentor junior engineers, enforce code and design standards, and drive adoption of reusability and best practices.
• Document technical designs, data flow diagrams, and data governance procedures.

Required Skills & Experience

• 6+ years of experience in Data Engineering, with focus on data warehousing, distributed processing, and cloud platforms.
• 4+ years of hands-on experience in Databricks: PySpark, Delta Lake
• Proficient in SQL, data modeling, and performance tuning for analytical workloads.
• Experience building scalable pipelines on Azure (ADF, Synapse, or Azure Functions).
• Understanding of data governance principles, RBAC, and metadata management.

Preferred / Bonus Skills

• Knowledge of Medallion architecture, CDC patterns, or event-driven data processing.
• Familiarity with Unity Catalog, Azure Purview, Databricks SQL, and data observability tools.

Why Join Us

• Be part of an innovation-driven team modernizing enterprise data ecosystems.
• Lead high-impact data initiatives with global customers across industries.
• Work with cutting-edge technologies like Databricks, Microsoft Fabric, and modern Azure stack.
• Accelerate your career in a cloud-native, AI-ready data engineering environment.