Databricks Data Engineer

Yantran; Chennai District; On-site; INR 900,000 - 1,400,000; Full time

Job summary

Yantran in India is seeking a Databricks Engineer to design and develop modern data engineering, analytics, and AI solutions on the Databricks platform.

You will build data pipelines, Lakehouse architectures, and notebooks with Python/PySpark/SQL, while implementing Delta Lake and CI/CD for deployments.

The role requires collaborating across cloud storage and enterprise systems, supporting ML workloads, and ensuring data quality and governance.

Qualifications

• Experience with the Databricks platform and Delta Lake.
• Strong PySpark and SQL coding skills in Python notebooks.
• Hands-on experience building and optimizing data pipelines.

Responsibilities

• Develop data pipelines using Databricks.
• Build Lakehouse architectures.
• Develop notebooks using Python/PySpark/SQL.
• Implement Delta Lake solutions.
• Build ETL/ELT pipelines.
• Optimize Spark workloads.
• Implement data quality and governance.
• Integrate Databricks with cloud storage and enterprise systems.
• Support machine learning and AI workloads.
• Implement CI/CD for Databricks deployments.

Skills

Databricks

Apache Spark/PySpark

Python

SQL

Delta Lake

Lakehouse architecture

Azure/AWS/GCP

Data pipelines

Unity Catalog

Databricks Workflows

CI/CD

Job description

Seeking a Databricks Engineer to design and develop modern data engineering, analytics, and AI solutions using the Databricks platform.

Responsibilities:

• Develop data pipelines using Databricks.
• Build Lakehouse architectures.
• Develop notebooks using Python/PySpark/SQL.
• Implement Delta Lake solutions.
• Build ETL/ELT pipelines.
• Optimize Spark workloads.
• Implement data quality and governance.
• Integrate Databricks with cloud storage and enterprise systems.
• Support machine learning and AI workloads.
• Implement CI/CD for Databricks deployments.

Required Skills:

• Databricks.
• Apache Spark/PySpark.
• Python.
• SQL.
• Delta Lake.
• Lakehouse architecture.
• Azure/AWS/GCP.
• Data pipelines.
• Unity Catalog preferred.
• Databricks Workflows.
• CI/CD.