Senior Data Engineer-Databricks Senior Data Engineer-Databricks Experience- 7 to 10 Years

Hoonartek; Pune District; On-site; INR 1,500,000 - 2,300,000; Full time

Job summary

Hoonartek is seeking a Senior Data Engineer – Databricks to design, develop, and optimize scalable data platforms using the Databricks Lakehouse architecture.

The candidate will build modern data pipelines, implement Delta Lake best practices, and leverage Auto Loader, SDP/DLT, Workflows, and Unity Catalog to deliver enterprise-grade data solutions on cloud platforms.

Qualifications

• Proficient in PySpark and SQL for large-scale processing.
• Experience with Delta Lake/Delta Tables and ACID transactions.
• Ingestion pipelines with Auto Loader and schema evolution.
• Hands-on SDP/DLT for declarative data pipelines.
• Orchestrating pipelines with Databricks Workflows.
• Knowledge of Unity Catalog for governance and metastore management.
• Understanding of data warehousing concepts and data marts.
• Experience building AI/BI dashboards (Databricks SQL Dashboards, Genie).
• Familiarity with cloud platforms (Azure/AWS/GCP) and cloud storage integration.

Responsibilities

• Design, develop, and optimize scalable data platforms using Databricks Lakehouse.
• Build modern data pipelines with best practices and Delta Lake.
• Mentor team members and deliver enterprise-grade data engineering solutions.
• Leverage Databricks capabilities including SDP, DLT, and Unity Catalog.
• Collaborate across teams to implement AI/BI dashboards and governance.

Skills

PySpark

SQL

Delta Lake

Auto Loader

SDP/DLT

Databricks Workflows

Unity Catalog

Data warehousing concepts

AI/BI dashboards

Cloud platforms (Azure/AWS/GCP)

Job description

We empower enterprises globally through intelligent, creative, and insightful services for data integration, data analytics and data visualization.

Hoonartek is a leader in enterprise transformation, data engineering and an acknowledged world-class Ab Initio delivery partner.

Using centuries of cumulative experience, research and leadership, we help our clients eliminate the complexities & risk of legacy modernization and safely deliver big data hubs, operational data integration, business intelligence, risk & compliance solutions and traditional data warehouses & marts.

At Hoonartek, we work to ensure that our customers, partners and employees all benefit from our unstinting commitment to delivery, quality and value. Hoonartek is increasingly the choice for customers seeking a trusted partner of vision, value and integrity

How We Work?

Define, Design and Deliver (D3) is our in-house delivery philosophy. It’s culled from agile and rapid methodologies and focused on ‘just enough design’. We embrace this philosophy in everything we do, leading to numerous client success stories and indeed to our own success.

We embrace change, empowering and trusting our people and building long and valuable relationships with our employees, our customers and our partners. We work flexibly, even adopting traditional/waterfall methods where circumstances demand it. At Hoonartek, the focus is always on delivery and value.

Job Description

We are seeking an experienced Senior Data Engineer – Databricks to design, develop, and optimize scalable data platforms using the Databricks Lakehouse architecture. The ideal candidate will have deep expertise in building modern data pipelines, implementing Delta Lake best practices, and leveraging the latest Databricks capabilities including Auto Loader, Spark Declarative Pipelines (SDP/DLT), Databricks Workflows, Unity Catalog, and AI/BI features.

The role requires strong hands-on development skills, the ability to mentor team members, and experience delivering enterprise-grade data engineering solutions on cloud platforms.

Job Requirement

• Strong proficiency in PySpark and SQL for large-scale data processing.
• Practical experience with Delta Lake / Delta Tables (ACID transactions, schema enforcement, versioning).
• Experience building ingestion pipelines with Auto Loader (cloudFiles, schema inference/evolution).
• Hands-on experience with SDP (Spark Declarative Pipelines / DLT) for building declarative data pipelines.
• Experience orchestrating pipelines with Databricks Workflows (jobs, tasks, triggers).
• Solid understanding of Unity Catalog for governance, permissions, and metastore management.
• Good grasp of data warehouse concepts — dimensional modeling, data marts, slowly changing dimensions.
• Experience building or supporting AI/BI dashboards (Databricks SQL Dashboards, Genie, or similar BI tools).
• Familiarity with cloud platforms (Azure/AWS/GCP) and cloud storage integration