Technical Lead, Data Engineering

Rakuten Kobo Inc.; Bengaluru; Hybrid; INR 2,000,000 - 4,200,000; Full time

Job summary

Rakuten Kobo Inc. in Bengaluru (Hybrid) seeks a Technical Lead for Data Engineering to own design, build, and operate large-scale data pipelines across PM/FM/IM/CM domains. You will work with ClickHouse for analytics and YugabyteDB for distributed SQL, enabling KPI dashboards and AI/ML workflows.

The role requires 6+ years in data platforms, strong SQL, and proficiency in Go/Java/Python/Scala, plus experience with Kafka, Flink or Spark and Kubernetes-based deployments.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering or a related field (or equivalent experience).
• 6+ years of software engineering experience building large-scale data platforms or distributed backend systems.
• Strong hands-on production experience with ClickHouse (or a comparable OLAP engine) including schema design and performance tuning.
• Strong SQL skills and experience with distributed SQL stores (YugabyteDB YBSQL, CockroachDB, or PostgreSQL-based).
• Proficiency in Go, Java, Python or Scala with production-grade engineering habits.
• Experience with stream and batch ingestion frameworks (Kafka, Flink, Spark).
• Experience running data infrastructure on Kubernetes and cloud environments (AWS/Azure/GCP).
• Track record of owning systems end-to-end: design, deployment, monitoring and production support.

Responsibilities

• Data pipeline architecture, ingestion, normalization and storage across multi-vendor sources.
• Design OLAP schemas, materialized views and TTL policies for time-series data.
• Model and operate distributed transactional stores for inventory and config data.
• Define vendor-independent AI-ready data models across PM/FM/IM/CM domains.
• Implement data quality, deduplication, lineage tracking and observability.
• Ensure horizontal scalability, fault tolerance and self-healing on Kubernetes.

Skills

Bachelor's degree or higher

6+ years experience

ClickHouse

YugabyteDB YBSQL

Distributed SQL

Go/Java/Python/Scala

Kafka/Flink/Spark

Kubernetes / cloud

End-to-end ownership

Education

Bachelor's or Master's degree in CS/Engineering

Tools

ClickHouse

YugabyteDB YBSQL

CockroachDB

Kafka

Flink

Spark

Kubernetes

Job description

Job Details

Job Title: Technical Lead, Data Engineering (RIO_OSS) Location: Bangalore (Hybrid)

Why should you choose us? Rakuten Symphony is reimagining telecom, changing supply chain norms and disrupting outmoded thinking that threatens the industry’s pursuit of rapid innovation and growth. Based on proven modern infrastructure practices, its open interface platforms make it possible to launch and operate advanced mobile services in a fraction of the time and cost of conventional approaches, with no compromise to network quality or security. Rakuten Symphony has operations in Japan, the United States, Singapore, India, South Korea, Europe, and the Middle East Africa region. For more information, visit: https://symphony.rakuten.com. Building on the technology Rakuten used to launch Japan’s newest mobile network, we are taking our mobile offering global. To support our ambitions to provide an innovative cloud-native telco platform for our customers, Rakuten Symphony is looking to recruit and develop top talent from around the globe. We are looking for individuals to join our team across all functional areas of our business – from sales to engineering, support functions to product development. Let’s build the future of mobile telecommunications together!

About Rakuten Group, Inc.

About Rakuten Group, Inc. (TSE: 4755) is a global leader in internet services that empower individuals, communities, businesses and society. Founded in Tokyo in 1997 as an online marketplace, Rakuten has expanded to offer services in e-commerce, fintech, digital content and communications to 2 billion members around the world. The Rakuten Group has over 30,000 employees, and operations in 30 countries and regions. For more information visit https://global.rakuten.com/corp/.

About the RIO Team

RIO (Rakuten Intelligent Operations) is Rakuten Symphony's AI-first operational intelligence platform and the OSS engineering team behind it. RIO replaces fragmented OSS tooling with a single, intelligent control layer for complex, multi-vendor telecom networks: unified real-time observability across RAN, Core, Transport and Cloud; AI-driven service assurance with anomaly detection, predictive analytics and proactive fault resolution; and closed-loop, intent-based automation.

What Do We Expect From You

As a Technical Lead on the Data Management Pipelines team, you will own the Design, build and operate large-scale pipelines around the existing data storage elements - that includes ingestion, normalization, storage and serving of operational data across the four classic OSS domains - Performance Management (PM), Fault Management (FM), Inventory Management (IM) and Configuration Management (CM). You will build high-throughput, low-latency pipelines on ClickHouse (OLAP / time-series analytics) and YugabyteDB YBSQL (distributed SQL), powering everything from KPI dashboards to the AI/ML models that will drive intelligent, closed-loop network operations.

Key Responsibilities

• Data pipeline architecture & development: Design, build and operate large-scale pipelines that ingest, normalize and store network telemetry, counters, KPIs, alarms, topology and configuration data from multi-vendor, multi-domain sources.
• ClickHouse engineering: Design OLAP schemas, materialized views and TTL/lifecycle policies for time-series and event data at billions of rows per day; optimize queries, merges and resource usage for sub-second analytical access.
• YBSQL / YugabyteDB engineering: Model and operate distributed transactional stores for inventory and configuration data - sharding, replication, geo-distribution, online schema evolution and performance tuning on a PostgreSQL-compatible distributed SQL layer.
• Cross-domain data models (PM/FM/IM/CM): Define vendor-independent, AI-ready data models across performance, fault, inventory and configuration domains; ensure consistent semantics, correlation keys and lineage across domains.
• Data quality & governance: Implement validation, deduplication, late-arrival handling, schema registry, lineage tracking and observability so downstream consumers (Data analytics, AI/ML, automation) can trust the data.
• Scale & reliability: Own throughput/latency SLOs for pipelines processing millions of events per second; build for horizontal scalability, graceful degradation and self-healing on Kubernetes-based telco cloud.
• Enable AI/ML: Partner closely with the AI Model/Harness team to expose clean, well-modeled, low-latency feature and training data from ClickHouse/YBSQL, and to productionize model outputs back into operational data flows.
• Operational excellence: Participate in on-call rotations, drive incident reviews, and continuously improve runbooks, automation and platform observability.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering or a related field (or equivalent experience).
• 6+ years of software engineering experience building large-scale data platforms or distributed backend systems.
• Strong hands-on production experience with ClickHouse (or a comparable OLAP engine such as Druid or Apache Pinot): schema design, materialized views, query and merge tuning, and capacity planning.
• Strong SQL skills and solid experience with a distributed SQL or NewSQL store (YugabyteDB YBSQL, CockroachDB, or comparable PostgreSQL-based distributed systems).
• Proficiency in at least one of: Go, Java, Python or Scala, with production-quality engineering habits.
• Deep understanding of data modeling for event, time-series and entity/state data; experience with stream and batch ingestion frameworks (e.g., Kafka, Flink, Spark).
• Experience running data infrastructure on Kubernetes and/or cloud environments (AWS/Azure/GCP/On-prem).
• Track record of owning systems end-to-end: design, deployment, monitoring and production support.

Preferred Qualifications

• Prior experience in telecom OSS/BSS, network management systems, or telemetry-heavy domains - ideally exposure to PM, FM, IM or CM data (3GPP counters, SNMP traps, syslog, NETCONF/YANG, topology).
• Hands-on YugabyteDB experience, including geo-partitioning, xCluster/geo-distribution and PostgreSQL wire-compatibility nuances.
• Experience building AI-ready data layers: feature stores, training data extraction, near-real-time serving for ML inference.
• Familiarity with data governance tooling (schema registries, lineage, catalogs) and event-streaming ecosystems.
• Experience with S3-compatible data lakes and lakehouse patterns.
• Contributions to open-source data infrastructure projects.

Rakuten Shugi Principles

Our worldwide practices describe specific behaviours that make Rakuten unique and united across the world. We expect Rakuten employees to model these 5 Shugi Principles of Success. Always improve, always advance. Only be satisfied with complete success - Kaizen. Be passionately professional. Take an uncompromising approach to your work and be determined to be the best. Hypothesize - Practice - Validate - Shikumika. Use the Rakuten Cycle to success in unknown territory. Maximize Customer Satisfaction. The greatest satisfaction for workers in a service industry is to see their customers smile. Speed!! Speed!! Speed!! Always be conscious of time. Take charge, set clear goals, and engage your team.

Rakuten Symphony is reimagining telecom, changing supply chain norms and disrupting outmoded thinking that threatens the industry’s pursuit of rapid innovation and growth. Based on proven modern infrastructure practices, its open interface platforms make it possible to launch and operate advanced mobile services in a fraction of the time and cost of conventional approaches, with no compromise to network quality or security. Rakuten Symphony has operations in Japan, the United States, Singapore, India, South Korea, Europe, and the Middle East Africa region. For more information, visit: https://symphony.rakuten.com Building on the technology Rakuten used to launch Japan’s newest mobile network, we are taking our mobile offering global. To support our ambitions to provide an innovative cloud-native telco platform for our customers, Rakuten Symphony is looking to recruit and develop top talent from around the globe. We are looking for individuals to join our team across all functional areas of our business – from sales to engineering, support functions to product development.

Let’s Build the Future

Let’s build the future of mobile telecommunications together!