Data bricks Engineer

Apolis Consulting; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Apolis Consulting is seeking an experienced Databricks Engineer in Bangalore to design and manage Databricks workflows, data pipelines, notebooks, and cluster optimization for production data engineering solutions.

The ideal candidate will have 5-8 years of hands-on Databricks experience, strong PySpark and Spark SQL skills, and familiarity with Delta Lake, Unity Catalog, and AWS integration. Face-to-face interview mode is preferred in Bengaluru.

Qualifications

• 5-8 years of relevant experience in Databricks / Data Engineering.
• Strong hands-on Databricks experience.
• Proficient in PySpark and Spark SQL.
• Hands-on with Delta Lake / Delta Tables.
• Experience with Databricks Jobs, Workflows and Pipelines.
• Experience with Unity Catalog and Databricks CI/CD pipelines.
• Exposure to Databricks Genie is preferred; certifications prioritized.

Responsibilities

• Develop and manage Databricks Jobs, Workflows and Data Pipelines.
• Work extensively with Delta Lake / Delta Tables.
• Develop data processing solutions using PySpark and Spark SQL.
• Create and maintain Databricks Notebooks and Dashboards.
• Implement and manage Unity Catalog.
• Handle workflow orchestration and scheduling.
• Perform Databricks cluster management, monitoring and optimization.
• Integrate Databricks with AWS S3, EC2 and IAM.
• Implement and maintain CI/CD pipelines.
• Work on data platform networking and security.
• Troubleshoot and debug Databricks jobs, pipelines and production issues.
• Work with Databricks Genie where applicable.

Skills

Databricks

Data Engineering

PySpark

Spark SQL

Delta Lake

AWS

Tools

Databricks Jobs

Unity Catalog

Job description

Job Title: Databricks Engineer

Location: Bangalore
Experience: 5-8 Years
Interview Mode: Face-to-Face (F2F)

Job Summary

We are looking for an experienced Databricks Engineer with strong hands-on experience in Databricks, PySpark, Spark SQL, Delta Lake and AWS. The candidate will be responsible for developing and managing Databricks workflows, data pipelines, notebooks, cluster optimization and production data engineering solutions.

Key Responsibilities

• Develop and manage Databricks Jobs, Workflows and Data Pipelines.


• Work extensively with Delta Lake / Delta Tables.


• Develop data processing solutions using PySpark and Spark SQL.


• Create and maintain Databricks Notebooks and Dashboards.


• Implement and manage Unity Catalog.


• Handle workflow orchestration and scheduling.


• Perform Databricks cluster management, monitoring and optimization.


• Integrate Databricks with AWS S3, EC2 and IAM.


• Implement and maintain CI/CD pipelines.


• Work on data platform networking and security.


• Troubleshoot and debug Databricks jobs, pipelines and production issues.


• Work with Databricks Genie where applicable.

Mandatory / Preferred Skills

• 58 years of relevant experience in Databricks / Data Engineering.


• Strong hands-on Databricks experience.


• Strong PySpark and Spark SQL skills.


• Hands-on experience with Delta Lake / Delta Tables.


• Experience with Databricks Jobs, Workflows and Pipelines.


• Experience with Unity Catalog.


• Good knowledge of AWS S3, EC2 and IAM.


• Experience in cluster management and performance optimization.


• Knowledge of CI/CD, networking and security.


• Strong debugging and troubleshooting skills.


• Databricks Certification preferred; certified candidates will be prioritized.


• Exposure to Databricks Genie is preferred.

Candidate Profile

Candidates should have strong hands-on experience building and managing Databricks-based data engineering solutions and should be comfortable working on production environments, troubleshooting issues and optimizing workloads.

Candidates based in or willing to work from Bangalore are preferred.

Interview: Face-to-Face (F2F)