JB001664 Azure Databricks Data Engineer Pan India · Hybrid

Techilaservices; India; Hybrid; INR 1,500,000 - 2,000,000; Full time

Job summary

Techilaservices in India seeks a data engineer to design, develop, and maintain scalable data solutions on Azure Databricks and related services. You will build enterprise pipelines, Lakehouse architectures, and data models, while integrating data from ERP/CRM and IoT sources.

The role emphasizes CI/CD practices, data governance, and collaboration with analysts and data scientists to support reporting and AI initiatives. Strong SQL and performance tuning skills are essential.

Qualifications

• Bachelor's degree in engineering or MCA or equivalent.
• Proficient in designing scalable data pipelines on Azure.
• Experience with Databricks, ADF and Delta Lake.

Responsibilities

• Design and implement end-to-end data pipelines on Azure.
• Develop ETL/ELT solutions with PySpark and Spark SQL.
• Build Lakehouse architectures with Bronze/Silver/Gold layers.
• Tune performance and ensure data quality and governance.
• Collaborate with stakeholders to support reporting and AI initiatives.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

SQL scripting

Performance Tuning

Data Warehousing Concepts

CI/CD

Git

Azure DevOps

Education

B.E./B.Tech/MCA or equivalent.

Tools

Azure Databricks

Azure Data Factory

ADLS Gen2

Delta Lake

SQL Server

Azure DevOps

Git

Spark

Power BI (analytics integration)

Job description

Role Summary

Design, develop, and maintain scalable data engineering solutions on the Azure platform using Azure Databricks, Azure Data Factory, ADLS Gen2, and related Azure data services. Build enterprise-grade data pipelines, transformation frameworks, and Lakehouse architectures.

Key Responsibilities

• Design and implement end-to-end data pipelines using Azure Databricks and Azure Data Factory.
• Develop scalable ETL/ELT solutions using PySpark and Spark SQL.
• Build and maintain Lakehouse architectures using Bronze, Silver and Gold layers.
• Develop Delta Lake-based data models and optimize data processing performance.
• Integrate data from ERP, CRM, databases, APIs, IoT, and external sources.
• Implement data quality checks, monitoring, and error-handling frameworks.
• Create and maintain CI/CD pipelines using Azure DevOps.
• Configure and manage Unity Catalog, data governance, and access controls.
• Work with Power BI and analytics teams to deliver curated data marts and semantic models.
• Collaborate with business stakeholders, architects, and data scientists to support reporting and AI initiatives.
• Troubleshoot production issues and optimize pipeline performance.
• Prepare technical documentation, deployment guides, and operational runbooks.

Required Technical Skills

• Azure Databricks
• PySpark and Spark SQL
• Azure Data Factory (ADF)
• Azure Data Lake Storage Gen2 (ADLS)
• Delta Lake
• SQL Server and SQL scripting
• Azure DevOps and CI/CD
• Git/Version Control
• Data Warehousing and ETL Concepts
• Performance Tuning and Optimization

Qualifications

• B.E./B.Tech/MCA or equivalent.