Azure Databricks Data Engineer

Larsen & Toubro Infotech Ltd (LTI)

Bengaluru

On-site

INR 2,500,000 - 4,500,000

Full time

7 days ago

Be an early applicant

Application generator

Get a reply from this employer — a resume and cover letter tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Larsen & Toubro Infotech Ltd (LTI) in Bengaluru seeks an experienced Azure Databricks Engineer to design, develop, optimize and support scalable cloud-based data platforms using Azure Data Services. You will build high-performance data pipelines and drive data modernization.

You will design ETL/ELT pipelines with Databricks, PySpark and Delta Lake, implement CI/CD with Azure DevOps, and collaborate with architects and stakeholders to ensure data quality and compliance.

Qualifications

7–11 years of IT/Data Engineering experience.
3 years of hands-on Azure Databricks implementation experience.
Experience building enterprise-scale data platforms on Azure.

Responsibilities

Design and implement scalable ETL/ELT pipelines using Azure Databricks and Azure Data Factory.
Develop data processing solutions using PySpark, Spark SQL and Python.
Build and maintain Delta Lake architectures and Databricks workflows.
Optimize performance of Spark jobs, clusters and notebooks.
Implement CI/CD pipelines using Azure DevOps for Databricks artifacts.
Work with ADLS Gen2, Azure Synapse and Azure SQL for enterprise data solutions.
Ensure data quality, governance, security and compliance standards.
Monitor, troubleshoot and support production data pipelines.
Collaborate with architects, business stakeholders and cross-functional teams.
Mentor junior engineers and participate in code reviews.

Skills

Strong analytical skills

Stakeholder management

Technical leadership

Agile delivery experience

Excellent communication

Tools

Azure Databricks

PySpark

Apache Spark

Azure Data Factory

ADF

Azure Data Lake Storage

ADLS Gen2

SQL

TSQL

Python

Delta Lake

Databricks Workflows

Azure DevOps

CI/CD

Data Modeling

Git

Version Control

Job description

Specialist - Data Engineering Role Summary Location-

Seeking an experienced Azure Databricks Engineer to design develop optimize and support scalable cloudbased data platforms using Azure Data Services The candidate will be responsible for building highperformance data pipelines implementing data engineering best practices and driving data modernization initiatives

Key Responsibilities

Design and implement scalable ETLELT pipelines using Azure Databricks and Azure Data Factory
Develop data processing solutions using PySpark Spark SQL and Python
Build and maintain Delta Lake architectures and Databricks workflows
Optimize performance of Spark jobs clusters and notebooks
Implement CICD pipelines using Azure DevOps for Databricks artifacts
Work with ADLS Gen2 Azure Synapse and Azure SQL for enterprise data solutions
Ensure data quality governance security and compliance standards
Monitor troubleshoot and support production data pipelines
Collaborate with architects business stakeholders and crossfunctional teams
Mentor junior engineers and participate in code reviews

Mandatory Skills