Senior Data Engineer

KSB; Pune District; Hybrid; INR 2,000,000 - 3,500,000; Full time

Job summary

KSB seeks a Senior Data Engineer to design, build, and scale modern data solutions. You will lead data architecture, pipelines, and cloud analytics, collaborating with engineering, analytics, product, and business teams to ensure reliable, high-quality data.

You will work with Python, Apache Spark, and Databricks, applying data lakehouse concepts, data governance, and scalable data processing. The role emphasizes autonomy and end‑to‑end ownership.

Qualifications

• Bachelor's degree in a relevant field.
• 3 years of project or coding experience in a company.
• Cloud experience with Databricks, AWS or Azure.
• Experience with CI/CD pipelines and deployment tooling.
• Strong Python and SQL skills.

Responsibilities

• Design and build scalable data pipelines and a growing data lake.
• Develop data processing workflows using Python, Spark, and Databricks.
• Define standards, best practices, and reusable data engineering frameworks.
• Ensure data quality, reliability, performance, and maintainability.
• Support data modeling, integration, and transformation for analytics.
• Drive automation, monitoring, and CI/CD improvements.
• Collaborate across teams as a technical interface for data platforms.
• Contribute to long-term data platform strategy.

Skills

Python programming

SQL

Data modeling

CI/CD

Autonomous worker

Clear communication

Education

Bachelor's degree in Software Engineering, Mathematics, Physics, or related field

Tools

Databricks

AWS

Azure

GitLab CI

IaC (Terraform, etc.)

Job description

About the Role

We are looking for a Senior Data Engineer to take ownership of designing, building, and scaling modern data solutions. In this role, you will provide technical leadership across data architecture, data pipelines, and cloud-based analytics platforms. You will work closely with engineering, analytics, product, and business teams to ensure reliable, high-quality, and accessible data across the organization.

Key Responsibilities

• Contribute to the design and development of scalable data pipelines and a growing data lake


• Build and extend data processing workflows using Python, Apache Spark, and Databricks


• Define technical standards, best practices, and reusable frameworks for data engineering


• Ensure data quality, reliability, performance, and maintainability across data solutions


• Support data modeling, data integration, and transformation processes for analytics and reporting


• Drive automation, monitoring, and CI/CD improvements to ensure operational excellence


• Collaborate across teams, acting as a technical interface between the data platform and engineering, analytics, and business stakeholders.


• Contribute to architecture decisions and long-term data platform strategy

Your Profile

• Outstanding programming experience, preferably in Python; ability to write clean, testable, production-grade code; able to write clean, testable, production-grade code


• Strong SQL skills and familiarity with structured and semi-structured data formats (JSON, Protobuf, Delta format)


• Hands-on experience with Apache Spark, ideally on Databricks, and understanding of the medallion architecture


• Solid grasp of data lakehouse principles, data modeling, and data governance concepts


• Experience building and maintaining CI/CD pipelines (e.g. GitLab CI); familiarity with IaC and deployment


• Cloud Platforms: Experience with AWS (cloud) or comparable cloud providers; familiarity with Databricks as a managed Lakehouse platform


• Experience with event-driven architectures or streaming platforms (e.g. Kafka)


• Proven track record deploying, monitoring, and maintaining data pipelines and services in
  production environments; experience with testing practices


• Able to work autonomously and take ownership of tasks end-to-end


• Clear and concise communicator comfortable working across engineering and data teams

Education & Experience

• Bachelor's degree in Software Engineering, Mathematics, Physics, or a related field


• 3 years of project/coding experience in a company


• Cloud experience:
  
  
  
  • Databricks, AWS/Azure
  
  
  • Staging, testing, Git, pipelines
  
  
  • Distributed systems, data pipelines
  
  
  
  


• Python experience:
  
  
  
  • Versioning, package management, requirements/environments
  
  
  • Change data capture
  
  
  
  


• Data engineering experience:
  
  
  
  • SQL
  
  
  • Partitioning
  
  
  • Data structures (tables, relations)
  
  
  • Lakehouse