Data Engineer, Databricks

Jobtailor; Bengaluru Urban; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Dysrupit in Bengaluru Urban, Karnataka is seeking an experienced Data Engineer with strong hands-on Databricks expertise. You will be responsible for designing, developing, and optimizing ELT pipelines for large-scale data solutions.

The ideal candidate will have proficiency in PySpark and SQL, and a good understanding of data Lakehouse architectures. This role requires on-site presence and alignment to UK business hours.

Qualifications

• Strong hands-on experience with Databricks and building ELT pipelines at scale.
• Proficient in PySpark and SQL for large-scale data processing.
• Good understanding of data Lakehouse architecture and best practices.

Responsibilities

• Design, develop, and maintain ELT pipelines using Databricks.
• Build and optimize large-scale data processing workflows.
• Collaborate with stakeholders to understand data requirements.

Skills

Databricks

PySpark

SQL

Data modeling techniques

Version control (Git)

Cloud platforms (Azure, AWS, GCP)

Education

Databricks Certified Associate Developer for Apache Spark

Microsoft Certified: Azure Data Engineer Associate

Tools

Apache Airflow

Azure Data Factory

Job description

JOB SUMMARY

We are seeking an experienced Data Engineer with strong hands‑on Databricks expertise to support the design, development, and optimization of enterprise data solutions. The successful candidate will have deep experience building and maintaining ELT pipelines, working with large‑scale data platforms, and applying modern data architecture principles. This role requires alignment to UK business hours and on‑site presence at our delivery center.

JOB RESPONSIBILITIES

• Design, develop, and maintain ELT pipelines using Databricks and related technologies.
• Build and optimize large‑scale data processing workflows using PySpark and SQL.
• Develop and manage data models to support analytics, reporting, and business intelligence requirements.
• Implement and support data Lakehouse architectures and data warehousing solutions.
• Collaborate with data analysts, architects, and business stakeholders to understand and deliver on data requirements.
• Ensure data quality, integrity, and governance across all data platforms.
• Monitor and troubleshoot data workflows, pipeline performance, and platform issues.
• Contribute to continuous improvement of data engineering standards and practices.
• Support and work during UK business hours as required.

REQUIRED TECHNICAL SKILLSET

• Strong hands‑on experience with Databricks (Delta Lake, Databricks Workflows, Unity Catalog).
• Proven experience building and supporting ELT pipelines at scale.
• Strong proficiency in PySpark and SQL.
• Experience with data modeling techniques and best practices.
• Good understanding of data Lakehouse architecture and data warehousing concepts.
• Experience with version control tools (e.g., Git).
• Familiarity with cloud platforms (Azure preferred; AWS or GCP considered).

PREFERRED EXPERIENCE

• Experience with orchestration tools such as Apache Airflow or Azure Data Factory.
• Exposure to data governance frameworks and tools (e.g., Unity Catalog, Microsoft Purview).
• Experience in enterprise consulting or regulated industry environments.
• Experience working with globally distributed teams across different time zones.

CERTIFICATIONS PREFERRED

• Databricks Certified Associate Developer for Apache Spark.
• Databricks Certified Data Engineer Associate or Professional.
• Microsoft Certified: Azure Data Engineer Associate (DP-203) or equivalent.
• Additional data or cloud certifications are a plus.