DataBricks Engineer

NLB Services; Dadri; On-site; INR 900,000 - 1,800,000; Full time

Job summary

NLB Services is seeking a Senior Databricks Engineer to design and implement scalable data pipelines on the Databricks Lakehouse Platform. You will build Delta Lake architectures, feature stores, and robust governance to power analytics, reporting, and machine learning at enterprise scale.

The role demands deep Databricks mastery, expert PySpark and Spark SQL, and hands-on cloud deployment experience across AWS, Azure, or GCP, with IAM and networking setup.

Qualifications

• 6+ years in data engineering with 3+ years Databricks in production.
• Deep knowledge of Databricks Runtime, Delta Lake, DLT, Unity Catalog, and Databricks Workflows.
• Expert level PySpark and Spark SQL with understanding of Spark execution model, lazy evaluation, and DAG optimization.
• Hands-on Databricks on AWS, Azure (ADLS Gen2), or GCP with IAM and networking setup.
• Strong Python for pipeline logic and automation scripts.
• Experience implementing data governance: masking, row-level security, audit logs.

Skills

Databricks

Delta Lake

Delta Live Tables

Unity Catalog

PySpark

Spark SQL

Python

SQL

Databricks Workflows

Apache Airflow

Azure Data Factory

AWS S3

Azure ADLS Gen2

GCP GCS

Row-level security

Audit logging

Tools

Databricks Runtime

Delta Lake

Delta Live Tables

Unity Catalog

Databricks Workflows

Apache Airflow

Azure Data Factory

Azure ADLS Gen2

AWS S3

GCP GCS

IAM configurations

Networking configurations

Job description

Job Description: Senior Databricks Engineer

We are seeking a Senior Databricks Engineer to design and implement scalable, high-performance data engineering solutions on the Databricks Lakehouse Platform. In this role, you will be responsible for building and maintaining robust data pipelines, Delta Lake architectures, and feature stores that power analytics, reporting, and machine learning at enterprise scale.

The ideal candidate combines deep expertise in Apache Spark with hands‑on Databricks platform mastery, and approaches every pipeline with a mindset that balances throughput performance with data quality guarantees and governance requirements.

Core Requirements

• Experience: 6+ years in Data Engineering with at least 3 years of hands‑on Databricks platform experience in a production environment.
• Databricks Expertise: Deep knowledge of Databricks Runtime, cluster management, Delta Lake, DLT, Unity Catalog, and Databricks Workflows.
• Spark Proficiency: Expert‑level PySpark and Spark SQL development, including understanding of the Spark execution model, lazy evaluation, and DAG optimization.
• Cloud Platforms: Hands‑on experience deploying Databricks on AWS, Azure (ADLS Gen2), or GCP with associated IAM and networking configurations.
• Python Development: Strong Python skills for pipeline logic, data transformations, utility libraries, and automation scripts.
• Data Governance: Experience implementing data governance controls on the Lakehouse including column masking, row‑level security, and audit logging.

Skill / AreaRequirement

Core PlatformDatabricks, Delta Lake, Delta Live Tables, Unity CatalogLanguagesPySpark, Spark SQL, Python, SQLCloud InfrastructureAzure (ADLS Gen2), AWS (S3), GCP (GCS)OrchestrationDatabricks Workflows, Apache Airflow, Azure Data FactoryGovernanceUnity Catalog, row‑level security, audit logging

More Info : [e-mail]