Senior Data Engineer

R Systems; Pune District; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

R Systems Senior Data Engineer role focusing on Databricks Lakehouse Platform; join a team to design, build, and optimize scalable data pipelines.

Responsibilities include PySpark/Delta Lake pipelines, Unity Catalog governance, and CI/CD deployment pipelines, with mentorship of teammates.

Qualifications

• 5+ years of experience in data engineering with significant hands-on work in Databricks.
• Strong proficiency in PySpark, Spark SQL, Delta Lake, and Delta Live Tables.
• Advanced skills in Python and SQL.
• Experience with Unity Catalog, cluster administration, and at least one major cloud platform (Azure, AWS, or GCP).
• Familiarity with orchestration tools such as Databricks Workflows or Airflow and Git-based CI/CD.
• Solid understanding of data warehousing and dimensional modeling.

Responsibilities

• Design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform.
• Partner with data science, analytics, and business teams to deliver high-performance data solutions.
• Build and maintain batch or streaming ETL pipelines using PySpark, Delta Lake, and Delta Live Tables.
• Manage Databricks clusters and implement Unity Catalog for governance.
• Design dimensional models, optimize Delta tables, and troubleshoot performance issues.
• Build deployment pipelines using CI/CD tools and support ML productionization while mentoring teammates.

Tools

Databricks

CI/CD tooling

Airflow

Cloud platforms

Cluster administration

Job description

We are looking for a Senior Data Engineer to join our team. In this role, you will design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will partner with data science, analytics, and business teams to deliver reliable, high-performance data solutions in a collaborative environment.

Your responsibilities will include building and maintaining batch or streaming ETL pipelines using PySpark, Delta Lake, and Delta Live Tables. You will manage Databricks clusters and implement Unity Catalog for governance. You will also design dimensional models, optimize Delta tables, and troubleshoot performance issues. A key part of the role involves building deployment pipelines using CI/CD tools and supporting ML productionization while mentoring fellow team members.

Required Skills:

• 5+ years of experience in data engineering with significant hands-on work in Databricks
• Strong proficiency in PySpark, Spark SQL, Delta Lake, and Delta Live Tables
• Advanced skills in Python and SQL
• Experience with Unity Catalog, cluster administration, and at least one major cloud platform (Azure, AWS, or GCP)
• Familiarity with orchestration tools such as Databricks Workflows or Airflow and Git-based CI/CD
• Solid understanding of data warehousing and dimensional modeling

Preferred Skills:

• Databricks Certified Data Engineer (Associate or Professional)