Senior Staff Engineer - Data Platform

Weekday; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Weekday is seeking a Senior Staff Engineer – Data Platform in Bengaluru to own the architecture and roadmap of a scalable data platform. You will lead Databricks-based solutions, build robust data pipelines, and mentor cross‑functional engineering teams.

The role requires 12–17 years of experience in data engineering or data platform development, deep knowledge of lakehouse architectures, Spark, Delta Lake, and cloud platforms, with strong collaboration and leadership skills.

Qualifications

• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Extensive experience building and scaling enterprise‑grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission‑critical data systems.
• Strong understanding of Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.

Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform.
• Databricks Engineering: Design, develop, and optimize Databricks‑based data solutions.
• Data Pipeline Development: Build and maintain batch and streaming data pipelines.
• Distributed Systems: Architect high‑throughput, fault‑tolerant data processing systems.
• Transactional Systems Integration: Design data integration between transactional and analytical systems.
• Performance Optimization: Improve processing, storage, and query performance.
• Technical Leadership: Lead engineering initiatives and establish best practices.
• Cross‑Functional Collaboration: Partner with product managers, analytics teams, and data scientists.

Skills

Data engineering

Databricks

Spark

Delta Lake

Lakehouse architecture

Python/Scala

Cloud platforms

SQL

Data governance

Leadership

Job description

This role is for one of Weekday’s clients

Min Experience: 12+ years

Location: Bengaluru, Karnataka, India, Bengaluru

JobType: full-time

We are looking for an experienced Senior Staff Engineer – Data Platform with 12–17 years of experience to lead the architecture, design, and evolution of our core data platform. This role will be responsible for building scalable, reliable, and high-performance data infrastructure spanning lakehouse architecture, analytical pipelines, and transactional systems. The ideal candidate will possess deep expertise in Data Engineering and Databricks, along with a strong understanding of distributed systems, cloud infrastructure, data governance, and modern data architecture. As a senior technical leader, you will drive architectural decisions, establish engineering standards, mentor technical teams, and collaborate with cross-functional stakeholders to deliver robust data solutions that support business-critical applications and advanced analytics.

Key Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform, ensuring scalability, reliability, performance, and long-term maintainability.
• Databricks Engineering: Design, develop, and optimize Databricks-based data solutions using Apache Spark, Delta Lake, and Lakehouse architecture principles.
• Data Pipeline Development: Build and maintain robust batch and streaming data pipelines to support analytics, reporting, machine learning, and operational use cases.
• Distributed Systems: Architect high-throughput, fault-tolerant data processing systems capable of handling large volumes of structured and unstructured data.
• Transactional Systems Integration: Design efficient data integration patterns between transactional databases, analytical platforms, and downstream data consumers.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, query execution, and resource utilization to improve system efficiency and control infrastructure costs.Data Governance and Quality: Establish standards for data quality, lineage, security, access control, metadata management, and regulatory compliance.
• Technical Leadership: Lead complex engineering initiatives, influence architectural decisions across teams, and establish best practices for data engineering and platform development.
• Cross-Functional Collaboration: Partner with product managers, analytics teams, data scientists, and engineering leaders to translate business requirements into scalable technical solutions.
• Mentorship and Innovation: Mentor senior engineers, conduct architecture reviews, promote engineering excellence, and evaluate emerging technologies to continuously improve platform capabilities.

Must-Have Skills

• Extensive experience in Data Engineering, data platform architecture, and large-scale data processing.
• Strong hands‑on expertise in Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.
• Proven experience designing and operating scalable ETL/ELT pipelines and distributed data systems.
• Strong understanding of data modeling, data warehousing, batch processing, and real‑time data streaming.
• Experience with cloud-based data platforms, preferably AWS, Microsoft Azure, or Google Cloud Platform.
• Proficiency in SQL and at least one programming language, such as Python or Scala.
• Strong knowledge of data security, governance, reliability, observability, and performance optimization.
• Demonstrated ability to lead architecture initiatives and influence technical strategy across multiple engineering teams.

Experience and Qualifications

• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Significant experience building and scaling enterprise‑grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission‑critical data systems.
• Excellent problem‑solving, communication, stakeholder management, and mentoring skills.