Senior Staff Engineer - Data Platform

Weekday AI (YC W21); Bengaluru; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Weekday AI is hiring a Senior Staff Engineer – Data Platform in Bengaluru to lead the architecture, design, and evolution of the core data platform. You will drive scalable, high-performance data infrastructure across lakehouse architectures, analytics pipelines, and transactional systems.

You will lead Databricks-based solutions, mentor engineers, and collaborate with product, analytics, and data science teams to deliver robust data capabilities for business-critical applications.

Qualifications

• 12–17 years of experience in software/data platform engineering.
• Experience building enterprise-grade data platforms in complex environments.
• Proven leadership and ability to drive architectural decisions.
• Excellent communication and stakeholder management.

Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and roadmap.
• Databricks Engineering: Design and optimize Spark-based solutions.
• Data Pipeline Development: Build batch and streaming pipelines.
• Distributed Systems: Architect high-throughput data processing.
• Transactional Systems Integration: Integrate transactional & analytical data flows.
• Performance Optimization: Improve processing, storage, and query efficiency.
• Data Governance and Quality: Establish data quality and lineage standards.
• Technical Leadership: Lead initiatives and set engineering best practices.
• Cross-Functional Collaboration: Align with product, analytics, and science teams.
• Mentorship and Innovation: Mentor engineers and evaluate new technologies.

Skills

Data engineering

Databricks

Apache Spark

Delta Lake

Lakehouse

ETL/ELT pipelines

Data modeling

Data warehousing

Batch processing

Real-time streaming

Cloud platforms

SQL

Python/Scala

Data security

Data governance

Observability

Performance optimization

Technical leadership

Architectural ownership

Job description

Job Description

This role is for one of Weekday’s clients

Min Experience: 12+ years
Location: Bengaluru, Karnataka, India, Bengaluru
JobType: full-time

We are looking for an experienced Senior Staff Engineer – Data Platform with 12–17 years of experience to lead the architecture, design, and evolution of our core data platform. This role will be responsible for building scalable, reliable, and high-performance data infrastructure spanning lakehouse architecture, analytical pipelines, and transactional systems. The ideal candidate will possess deep expertise in Data Engineering and Databricks, along with a strong understanding of distributed systems, cloud infrastructure, data governance, and modern data architecture. As a senior technical leader, you will drive architectural decisions, establish engineering standards, mentor technical teams, and collaborate with cross-functional stakeholders to deliver robust data solutions that support business-critical applications and advanced analytics.

Key Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform, ensuring scalability, reliability, performance, and long-term maintainability.
• Databricks Engineering: Design, develop, and optimize Databricks-based data solutions using Apache Spark, Delta Lake, and Lakehouse architecture principles.
• Data Pipeline Development: Build and maintain robust batch and streaming data pipelines to support analytics, reporting, machine learning, and operational use cases.
• Distributed Systems: Architect high-throughput, fault-tolerant data processing systems capable of handling large volumes of structured and unstructured data.
• Transactional Systems Integration: Design efficient data integration patterns between transactional databases, analytical platforms, and downstream data consumers.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, query execution, and resource utilization to improve system efficiency and control infrastructure costs.
• Data Governance and Quality: Establish standards for data quality, lineage, security, access control, metadata management, and regulatory compliance.
• Technical Leadership: Lead complex engineering initiatives, influence architectural decisions across teams, and establish best practices for data engineering and platform development.
• Cross-Functional Collaboration: Partner with product managers, analytics teams, data scientists, and engineering leaders to translate business requirements into scalable technical solutions.
• Mentorship and Innovation: Mentor senior engineers, conduct architecture reviews, promote engineering excellence, and evaluate emerging technologies to continuously improve platform capabilities.

Must‑Have Skills

• Extensive experience in Data Engineering, data platform architecture, and large-scale data processing.
• Strong hands‑on expertise in Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.
• Proven experience designing and operating scalable ETL/ELT pipelines and distributed data systems.
• Strong understanding of data modeling, data warehousing, batch processing, and real‑time data streaming.
• Experience with cloud‑based data platforms, preferably AWS, Microsoft Azure, or Google Cloud Platform.
• Proficiency in SQL and at least one programming language, such as Python or Scala.
• Strong knowledge of data security, governance, reliability, observability, and performance optimization.
• Demonstrated ability to lead architecture initiatives and influence technical strategy across multiple engineering teams.

Experience and Qualifications

• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Significant experience building and scaling enterprise‑grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission‑critical data systems.
• Excellent problem‑solving, communication, stakeholder management, and mentoring skills.

Requirements