Databricks Data Engineer — Pipelines, Delta Lake & CI/CD

LTM; Raritan (NJ); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical plan

Disability coverage

401(k) plan

Life insurance

Vacation & paid leave

Paternity/maternity leave

Job summary

LTIMindtree is seeking a Data Engineer to design and implement scalable data pipelines using Databricks. You will work with Delta Lake, Unity Catalog, and Delta Sharing to ensure secure, governed data platforms. Strong Python and PySpark skills are essential to deliver robust ETL/ELT solutions for cross-functional teams.

You will partner with architects and analysts to meet data requirements, optimize performance, and support production deployments with modern DevOps practices.

Qualifications

• Experience with Databricks and Delta Lake technologies.
• Hands-on coding in Python and PySpark for large-scale data processing.
• Strong SQL understanding and query optimization.
• Experience implementing CI/CD for Databricks workloads.
• Familiarity with Git-based source control and DevOps pipelines.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks notebooks.
• Build and optimize data processing solutions with PySpark and Python.
• Implement and manage data governance, security, and access with Unity Catalog.
• Work with Delta Lake features including Delta Sharing, Time Travel, and CDF.
• Develop and maintain CI/CD pipelines for Databricks deployments.
• Tune performance of Databricks workloads and monitor data quality.

Skills

Databricks

Python

PySpark

SQL

CI/CD

Git

Delta Lake

Unity Catalog

Delta Sharing

Structured Streaming

Data governance

Medallion Architecture

Large-scale processing

Spark optimization

DevOps practices

Tools

Databricks

Delta Lake

Delta Sharing

Unity Catalog

Job description

LTIMindtree is seeking a Data Engineer to design and implement scalable data pipelines using Databricks. You will work with Delta Lake, Unity Catalog, and Delta Sharing to ensure secure, governed data platforms. Strong Python and PySpark skills are essential to deliver robust ETL/ELT solutions for cross-functional teams.

You will partner with architects and analysts to meet data requirements, optimize performance, and support production deployments with modern DevOps practices.