Senior Data Engineer / SSE

JobCubby; Bengaluru; On-site; INR 2,500,000 - 4,500,000; Full time

Job summary

Apna is seeking a Senior Data Engineer to design, build, and operate scalable data pipelines and lakehouse architecture that power analytics, product intelligence, and ML workflows.

You will own data reliability, quality, and performance across large-scale datasets, collaborate with cross-functional teams, and mentor engineers. This role is based at our Bengaluru office, with 5 days a week on-site commitments.

Qualifications

• 4-6+ years of data engineering experience with large-scale data platforms.
• Strong SQL skills and proficiency in Python, Java, or Scala.
• Hands-on experience with Apache Airflow and distributed query engines (Presto/Trino).
• Experience designing lakehouse architectures with upserts, schema evolution, and partitioning.
• Ability to build reliable ETL/ELT pipelines and improve data quality.

Responsibilities

• Design, build, and operate scalable data pipelines and lakehouse architecture.
• Drive engineering standards for data modeling, partitioning, and lineage.
• Mentor data engineers and ensure platform reliability and performance.
• Collaborate with product analytics and ML teams to satisfy data needs.
• Improve observability, SLA tracking, and cost-efficient data processing.

Skills

Data engineering

SQL

Python/Java/Scala

Distributed query engines

Airflow

Tools

Apache Airflow

Presto / Trino

Apache Hudi

Kafka

Spark

Flink

Hive

Iceberg

Delta Lake

Job description

to apply - email only, no card. You can also save this posting or score it againstyour profile with AI.## About the roleYou will be responsible for designing, building, and operating scalable data pipelines and lakehouse architecture to support product intelligence and machine learning. Additionally, you will drive engineering standards for data modeling and mentor team members to ensure platform reliability and performance.## RequirementsThe role requires 4-6 years of experience in data engineering with strong proficiency in distributed query engines and orchestration systems like Apache Airflow. Candidates must possess deep knowledge of data architecture, SQL, and programming languages such as Python, Java, or Scala.## Full descriptionRole: Senior Data Engineer / SSERequirement: 1Team: Data Platform / EngineeringLocation: Work from Office - Domlur, Bangalore (5 days / week)Experience: 4-6 Years of ExperienceWhy Join ApnaAt Apna, data is central to how we build products, understand users, improve employer outcomes, power recommendations, and scale decision-making. This role gives you the opportunity to build the backbone of Apna's data platform and influence how data is used across the company.You will work on real-world, high-scale problems across jobs, users, employers, communities, matching, growth, and AI-driven systems.About the RoleApna is looking for a Senior Software Engineer to build and scale our core data platform. This role will work on large-scale data pipelines, lakehouse architecture, query platforms, workflow orchestration, and data reliability systems that power analytics, product intelligence, machine learning, business dashboards, experimentation, and operational decision-making across Apna.We are looking for someone who can think deeply about data architecture, design reliable pipelines, improve data quality, and help build a platform that can scale with Apna's growth.What You'll Own:You will be responsible for designing, building, and operating critical parts of Apna's data platform, including:* Building scalable batch and near-real-time data pipelines across product, business, growth, and ML use cases.* Designing and improving our lakehouse architecture using technologies likeApache Hudi.* Working with query engines such asPresto / Trinofor large-scale analytical workloads.* Building and maintaining orchestration workflows usingApache Airflow.* Creating reusable data models, curated datasets, and reliable data marts for analytics and product teams.* Improving data platform reliability, observability, SLA tracking, lineage, and data quality checks.* Optimizing storage, compute, query performance, and pipeline costs.* Partnering with product, analytics, ML, and backend engineering teams to understand data needs and convert them into scalable platform solutions.* Driving engineering standards around data modeling, schema evolution, partitioning, deduplication, backfills, replayability, and pipeline ownership.* Mentoring data engineers and influencing architecture decisions across teams.What We're Looking ForMust Have* Strong experience indata engineering, preferably at scale.* Hands-on experience withApache Airflowor similar orchestration systems.* Strong knowledge ofPresto / Trinoor other distributed query engines.* Good understanding ofApache Hudiconcepts such as:* Copy-on-write vs merge-on-read* Upserts and deletes* Incremental reads* Compaction* Clustering* Timeline and commits* Schema evolution* Partitioning strategy* Strong knowledge of distributed data processing and storage systems.* Ability to design and build reliable ETL / ELT pipelines.* Strong SQL skills and ability to debug complex data issues.* Good understanding of different data architectures, including:* Data warehouse* Data lake* Lakehouse* Lambda architecture* Kappa architecture* Medallion architecture* Event-driven data architecture* Experience with data modeling for analytics and reporting.* Strong programming skills in at least one language such asPython, Java, or Scala.* Ability to reason about trade-offs between freshness, cost, reliability, latency, and complexity.* Strong debugging and production ownership mindset.Good to Have* Experience with Kafka, Spark, Flink, Hive, Iceberg, Delta Lake, or BigQuery.* Experience building internal data platforms or self-serve data infrastructure.* Experience with data quality frameworks such as Great Expectations, Deequ, Soda, or custom validation systems.* Exposure to ML feature pipelines or feature stores.* Experience with metadata management, data catalogs, lineage, and governance.* Experience with cloud infrastructure such as AWS, GCP, or Azure.* Understanding of privacy, compliance, PII handling, and access control in data systems.What Success Looks Like In this role, success means:* Critical business and product datasets are reliable, discoverable, and trusted.* Pipelines are observable, recoverable, and have clear SLAs.* Query performance improves across major analytical workloads.* Data freshness and quality issues reduce significantly.* Teams can build on top of the data platform faster without reinventing pipelines.* The platform can scale with Apna's user, job, employer, and engagement data.