Databricks Consultant

Doehler; Pune District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Doehler is seeking a skilled data engineer to own the implementation, quality, and operation of data products on the Doehler data platform in Pune. You will develop data products based on the Shared Data Foundation and ensure high engineering standards through tests, code reviews, CI/CD, monitoring, and incident analysis.

The role requires collaboration with Architecture, Platform Engineering, and Data Governance, plus experience with Databricks, Azure, Python, SQL, and data modeling.

Qualifications

• Degree in Computer Science, Business Information Systems, Data Science, Software Engineering, or a comparable qualification.
• Several years of experience in data or software engineering and/or with modern data platforms.
• Very good knowledge of Databricks and cloud data platforms, ideally Microsoft Azure.
• Experience with robust data pipelines, ETL/ELT, and batch and streaming processing.

Responsibilities

• Responsibility for the implementation, quality, and operation of data products on Dhlers data platform.
• Development of data products based on the Shared Data Foundation.
• Ensuring high engineering standards through testing, code reviews, CI/CD, monitoring, and incident analysis.
• Further development of standards, templates, and components for integration, transformation, orchestration, and documentation.
• Implementing scalable, high-performance, and maintainable solutions in Databricks.
• Close collaboration with Architecture, Platform Engineering, and Data Governance.
• Providing technical support to Domain Leads with requirements, estimations, solution design, and go-live activities.
• Responsibility for technical documentation, runbooks, operational handovers, and the stability of productive data products.

Skills

Databricks

Azure

Python

SQL

Data modeling

Data governance

CI/CD pipelines

Git & code reviews

Observability

Stakeholder comms

Project mgmt

Education

Bachelor's degree in CS or related

Tools

Databricks

Azure

dbt

Unity Catalog

Git

CI/CD tooling

Job description

Your role

• Responsibility for the implementation, quality, and operation of data products on Dhlers data platform.
• Development of data products based on the Shared Data Foundation.
• Ensuring high engineering standards through testing, code reviews, CI/CD, monitoring, and incident analysis.
• Further development of standards, templates, and components for integration, transformation, orchestration, and documentation.
• Implementing scalable, high-performance, and maintainable solutions in Databricks.
• Close collaboration with Architecture, Platform Engineering, and Data Governance.
• Providing technical support to Domain Leads with requirements, estimations, solution design, and go-live activities.
• Responsibility for technical documentation, runbooks, operational handovers, and the stability of productive data products.

Your profile

• Degree in Computer Science, Business Information Systems, Data Science, Software Engineering, or a comparable qualification.
• Several years of experience in data or software engineering and/or with modern data platforms.
• Very good knowledge of Databricks and cloud data platforms, ideally Microsoft Azure.
• Experience with robust data pipelines, ETL/ELT, and batch and, where applicable, streaming processing.
• Very good knowledge of Python and SQL; PySpark and/or Spark SQL is an advantage.
• Experience with data modeling and reusable, consumption-oriented data models.
• Knowledge of dbt or comparable frameworks for transformation, testing, and documentation is desirable.
• Experience with Git, pull requests, code reviews, and CI/CD pipelines.
• Expertise in data quality, observability, monitoring, logging, error handling, and performance optimization.
• Knowledge of the SAP environment, particularly S/4HANA, as well as the integration of SAP and non-SAP data.
• Understanding of data governance, e.g. access control, metadata, lineage, data catalogs, and sensitivity classification; experience with Unity Catalog is a plus.
• Experience managing external developers or service providers is desirable.
• A structured, pragmatic, and hands-on approach to work, along with strong communication skills.