Senior Databricks Engineer

Prodapt

Bengaluru

On-site

INR 3,500,000 - 5,500,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Prodapt seeks a Senior Databricks Engineer to lead the design and implementation of modern cloud-based data platforms. The role focuses on Databricks, Snowflake, AWS, and enterprise data architecture, enabling scalable, secure, and high-performance data solutions across the organization.

You will guide data strategy, drive analytics and AI/ML workloads, and provide hands-on architecture leadership while collaborating with data scientists, architects, and engineering teams to ensure governance,

Qualifications

  • 12–16 years of experience in Data Engineering, Data Architecture, and Cloud Data Platforms.
  • Strong expertise in Databricks, Snowflake, and AWS Cloud Services.
  • Hands-on experience designing and delivering large-scale enterprise data architectures and modern data lakehouse solutions.

Responsibilities

  • Design and architect scalable, secure, and highly available enterprise data platforms using Databricks, Snowflake, and AWS Cloud.
  • Lead development and optimization of large-scale ETL/ELT pipelines for batch and real-time processing.
  • Architect and deploy AWS-based data platform components ensuring IAM, security, governance, and networking compliance.
  • Develop real-time data processing solutions for low-latency use cases using modern streaming technologies.
  • Design scalable data models using Kimball, Inmon, Star Schema, and Snowflake Schema approaches.

Skills

Databricks
Snowflake
AWS Cloud Services
ETL/ELT pipelines
Real-time data processing
SQL
Python
Spark
Databricks APIs
Structured Streaming
Data governance
Observability
Stakeholder management
Mentorship

Education

Bachelor's degree in a related field

Tools

Apache Kafka
Airflow
dbt
Iceberg
Kubernetes

Job description

Overview

Prodapt is seeking a highly experienced Senior Databricks Engineer to lead the design and implementation of modern cloud-based data platforms. The ideal candidate will have deep expertise in Databricks, Snowflake, AWS, Data Engineering, and Enterprise Data Architecture, with proven experience building scalable, secure, and high‑performance data solutions. This role requires strong technical leadership, hands‑on architecture experience, and the ability to drive enterprise‑wide data modernization initiatives while supporting advanced analytics, AI, and machine learning workloads.

Responsibilities
  • Design and architect scalable, secure, and highly available enterprise data platforms using Databricks, Snowflake, and AWS Cloud.
  • Lead the development and optimization of large-scaleETL/ELT pipelines, supporting both batch and real‑time data processing workloads.Design and implement modern data lakehouse architectures and cloud‑native data ecosystems.
  • Architect and deploy AWS‑based data platform components while ensuring compliance with enterprise security, IAM, governance, and networking standards.Build and optimize data ingestion frameworks integratingDatabricks APIs, REST APIs, GraphQL APIs, and third‑party data sources.
  • Develop real‑time data processing solutions for business‑critical, low‑latency use cases using modern streaming technologies.
  • Design and implement scalable data models, incorporating both traditional data warehousing methodologies (Kimball, Inmon, Star Schema, Snowflake Schema) and modern semi‑structured/unstructured data patterns.
  • Collaborate with business stakeholders, architects, data scientists, and engineering teams to define and execute data strategies.Drive performance optimization, cost management, scalability, and reliability initiatives across the data platform landscape.
  • Provide technical leadership, architectural governance, code reviews, and mentorship to engineering teams.
  • Support AI and Machine Learning initiatives by designing data foundations that enableLLMs, Generative AI, Machine Learning pipelines, and advanced analytics solutions.
  • Establish best practices for data engineering, architecture, governance, observability, and platform operations.Prepare and maintain architecture documentation, design standards, and technical roadmaps.
Requirements
  • 12 to 16 years of experience in Data Engineering, Data Architecture, and Cloud Data Platforms.
  • Strong expertise in Databricks, Snowflake, and AWS Cloud Services.
  • Proven experience designing and delivering large‑scale enterprise data architectures and modern data lakehouse solutions.
  • Hands‑on experience building and optimizing ETL/ELT pipelines, data ingestion frameworks, and distributed data processing systems.
  • Strong knowledge of real‑time and batch data processing architectures.
  • Expertise in Databricks Workflows, Delta Lake, Spark, Structured Streaming, and Databricks APIs.
  • Experience integrating REST and GraphQL APIs for enterprise‑scale data ingestion and orchestration.
  • Strong understanding of Data Warehousing concepts including Kimball, Inmon, Star Schema, and Snowflake Schema methodologies.
  • Experience designing scalable and secure solutions within the AWS ecosystem, including IAM, networking, and governance controls.
  • Strong proficiency in SQL, Python, and Spark‑based development.
  • Experience with performance tuning, optimization, data quality, metadata management, and platform observability.
  • Ability to provide technical leadership, solution architecture guidance, and mentorship to engineering teams.
  • Excellent stakeholder management, communication, problem‑solving, and decision‑making skills.
Preferred Qualifications

Experience with modern data ecosystem tools such as Apache Kafka, Airflow, dbt, Iceberg, and Kubernetes. Exposure to Generative AI, LLMs, Machine Learning platforms, MLOps, and advanced analytics architectures will be highly preferred. Experience with data governance, security, compliance frameworks, and enterprise‑scale cloud modernization programs is advantageous. Relevant certifications in Databricks, Snowflake, AWS Solutions Architecture, or Data Engineering will be considered a strong advantage. Candidates with experience supporting AI‑ready data platforms and leading enterprise data transformation initiatives will be highly valued.

Get your free, confidential resume review.

or drag and drop your file here.