Sr Databricks Engineer

Kani Solutions Inc; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Kani Solutions Inc in Bengaluru, India seeks a seasoned Senior Data Engineer specialized in Databricks to expand our enterprise Databricks data platform. You will focus on integrating datasets, engineering scalable pipelines, and building new business dimensions for analytical use.

Responsibilities include developing scalable data solutions, applying best practices for governance, and delivering performance-tuned data assets for reporting and downstream applications.

Qualifications

• 8+ years of professional experience in Data Engineering.
• 4+ years hands-on with Databricks.
• Strong proficiency in Databricks, PySpark, Python, SQL.
• Unity Catalog and Databricks Asset Bundles experience.
• Proven experience designing enterprise ETL/ELT solutions.
• Dimensional modeling, KPIs, and analytical datasets.
• CI/CD pipelines with GitHub/GitLab.

Responsibilities

• Architect and build scalable data pipelines using Databricks and PySpark.
• Incorporate additional data sources into Databricks Lakehouse.
• Create dimensional models, business dimensions, measures, and datasets for analytics.
• Prepare and optimize data to support reporting and downstream apps.
• Contribute to data quality, validation, monitoring, and performance tuning.
• Investigate and resolve platform-related issues.

Skills

Databricks

PySpark

Python

SQL

Unity Catalog

Asset Bundles

Security IAM

ETL/ELT

CI/CD pipelines

GitHub

GitLab

Dimensional modeling

Cloud Data

Tools

GitHub

GitLab

Power BI

Job description

We are looking for a seasoned Senior Data Engineer with strong Databricks expertise to contribute to the expansion of our enterprise Databricks data platform. The selected candidate will focus on integrating additional datasets, engineering scalable data pipelines, and building new business dimensions and measures within the existing Databricks ecosystem.

Key Responsibilities:

• Architect and build scalable data pipelines leveraging Databricks and PySpark
• Connect and incorporate additional data sources into the existing Databricks Lakehouse environment
• Create and refine dimensional models, business dimensions, measures, and curated datasets for analytical use
• Prepare, transform, and optimize data to support reporting, analytics, and downstream applications
• Contribute to data quality initiatives, validation activities, monitoring, and performance tuning
• Investigate and resolve platform-related issues throughout the project.

Platform & Data Management

• Apply Databricks development, deployment, security, and governance best practices across solutions
• Use Unity Catalog to address data governance and access control requirements
• Build, package, and deploy solutions through Databricks Asset Bundles
• Work closely with the project team to convert business and technical requirements into scalable data solutions.

DevOps & Automation

• Establish and manage CI/CD workflows for data engineering components
• Use GitHub and/or GitLab to manage source code, branching approaches, and peer code reviews
• Contribute to automated deployment workflows and release management activities.

Required Skills:

• 8+ years of professional experience in Data Engineering
• 4+ years of practical, hands‑on experience with Databricks (required)
• Strong proficiency with:
• Databricks
• PySpark
• Python
• SQL
• Unity Catalog
• Databricks Asset Bundles
• Databricks Security & Access Management
• Proven experience designing and supporting enterprise‑level ETL/ELT solutions
• Extensive experience developing dimensional models, business dimensions, measures, KPIs, and analytical datasets
• Hands‑on experience developing and maintaining CI/CD pipelines
• Strong background in source code management using GitHub and/or GitLab
• Experience delivering solutions within cloud‑based data environments
• Strong verbal and written communication abilities.

Preferred Skills:

• Exposure to Fivetran or comparable cloud‑based data ingestion solutions
• Working knowledge of REST APIs and integrations with enterprise applications
• Experience with Power BI or comparable enterprise business intelligence and reporting platforms
• Background supporting AI, machine learning, or advanced analytics projects
• Experience working with HR, Finance, or other enterprise business data domains.