Technical Lead, Data Engineering

Rakuten Symphony; Bengaluru; On-site; INR 2,500,000 - 5,000,000; Full time

Job summary

Rakuten Symphony seeks a Technical Lead, Data Engineering (RIO_OSS) to own design, build and operate large-scale data pipelines. You will manage ingestion, normalization, storage and serving of operational data across PM/FM/IM/CM domains using ClickHouse and YugabyteDB YBSQL, powering dashboards and AI/ML models.

You will shape cross-domain data models, ensure data quality and governance, and drive scalable, low-latency data infrastructure on Kubernetes in a cloud-friendly telco environment.

Qualifications

• 6+ years of software engineering in large-scale data platforms or distributed backends.
• Hands-on with ClickHouse; schema design, materialized views and performance tuning.
• Strong SQL and distributed SQL or NewSQL experience (YugabyteDB YBSQL, CockroachDB or PostgreSQL-based).
• Proficiency in Go, Java, Python or Scala with production-grade habits.
• Experience with Kafka, Flink or Spark for streaming and batch processing.
• Experience on Kubernetes and cloud/on-prem environments.

Responsibilities

• Design, build and operate large-scale data pipelines for telemetry, KPIs and topology data.
• Develop ClickHouse schemas, materialized views and TTL policies for time-series data.
• Model and operate distributed transactional stores with YugabyteDB YBSQL; ensure performance.
• Define vendor-independent data models across PM/FM/IM/CM domains.
• Implement data quality, lineage tracking and observability for downstream consumers.
• Own throughput/latency SLOs; ensure horizontal scalability and resilience on telco cloud.
• Collaborate with AI/ML teams to expose feature data for models and automation.
• Participate in on-call rotations and improve runbooks and monitoring.

Skills

ClickHouse

YugabyteDB

SQL

Go/Java/Python/Scala

Kubernetes

Data modeling

Distributed systems

Kafka/Flink/Spark

Education

Bachelor's or Master's in CS/Engineering

Tools

Druid

Apache Pinot

Kafka

Flink

Spark

AWS/Azure/GCP

Job description

Job Title: Technical Lead, Data Engineering (RIO_OSS)

Why should you choose us?

Rakuten Symphony is reimagining telecom, changing supply chain norms and disrupting outmoded thinking that threatens the industry’s pursuit of rapid innovation and growth. Based on proven modern infrastructure practices, its open interface platforms make it possible to launch and operate advanced mobile services in a fraction of the time and cost of conventional approaches, with no compromise to network quality or security.

Rakuten Symphony has operations in Japan, the United States, Singapore, India, South Korea, Europe, and the Middle East Africa region. For more information, visit: https://symphony.rakuten.com.

Building on the technology Rakuten used to launch Japan’s newest mobile network, we are taking our mobile offering global.

To support our ambitions to provide an innovative cloud-native telco platform for our customers, Rakuten Symphony is looking to recruit and develop top talent from around the globe. We are looking for individuals to join our team across all functional areas of our business – from sales to engineering, support functions to product development.

Let’s build the future of mobile telecommunications together!

About Rakuten Group, Inc. (TSE: 4755) is a global leader in internet services that empower individuals, communities, businesses and society. Founded in Tokyo in 1997 as an online marketplace, Rakuten has expanded to offer services in e-commerce, fintech, digital content and communications to 2 billion members around the world. The Rakuten Group has over 30,000 employees, and operations in 30 countries and regions. For more information visit https://global.rakuten.com/corp/.

About the RIO Team

RIO (Rakuten Intelligent Operations) is Rakuten Symphony's AI-first operational intelligence platform and the OSS engineering team behind it. RIO replaces fragmented OSS tooling with a single, intelligent control layer for complex, multi-vendor telecom networks: unified real-time observability across RAN, Core, Transport and Cloud; AI-driven service assurance with anomaly detection, predictive analytics and proactive fault resolution; and closed-loop, intent-based automation.

What Do We Expect From You

As a Technical Lead on the Data Management Pipelines team, you will own the Design, build and operate large-scale pipelines around the existing data storage elements - that includes ingestion, normalization, storage and serving of operational data across the four classic OSS domains - Performance Management (PM), Fault Management (FM), Inventory Management (IM) and Configuration Management (CM). You will build high-throughput, low-latency pipelines on ClickHouse (OLAP / time-series analytics) and YugabyteDB YBSQL (distributed SQL), powering everything from KPI dashboards to the AI/ML models that will drive intelligent, closed-loop network operations

Key Responsibilities

• Data pipeline architecture & development: Design, build and operate large-scale pipelines that ingest, normalize and store network telemetry, counters, KPIs, alarms, topology and configuration data from multi-vendor, multi-domain sources.
• ClickHouse engineering: Design OLAP schemas, materialized views and TTL/lifecycle policies for time-series and event data at billions of rows per day; optimize queries, merges and resource usage for sub-second analytical access.
• YBSQL / YugabyteDB engineering: Model and operate distributed transactional stores for inventory and configuration data - sharding, replication, geo-distribution, online schema evolution and performance tuning on a PostgreSQL-compatible distributed SQL layer.
• Cross-domain data models (PM/FM/IM/CM): Define vendor-independent, AI-ready data models across performance, fault, inventory and configuration domains; ensure consistent semantics, correlation keys and lineage across domains.
• Data quality & governance: Implement validation, deduplication, late-arrival handling, schema registry, lineage tracking and observability so downstream consumers (Data analytics, AI/ML, automation) can trust the data.
• Scale & reliability: Own throughput/latency SLOs for pipelines processing millions of events per second; build for horizontal scalability, graceful degradation and self-healing on Kubernetes-based telco cloud.
• Enable AI/ML: Partner closely with the AI Model/Harness team to expose clean, well-modeled, low-latency feature and training data from ClickHouse/YBSQL, and to productionize model outputs back into operational data flows.
• Operational excellence: Participate in on-call rotations, drive incident reviews, and continuously improve runbooks, automation and platform observability.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering or a related field (or equivalent experience).
• 6+ years of software engineering experience building large-scale data platforms or distributed backend systems.
• Strong hands-on production experience with ClickHouse (or a comparable OLAP engine such as Druid or Apache Pinot): schema design, materialized views, query and merge tuning, and capacity planning.
• Strong SQL skills and solid experience with a distributed SQL or NewSQL store (YugabyteDB YBSQL, CockroachDB, or comparable PostgreSQL-based distributed systems).
• Proficiency in at least one of: Go, Java, Python or Scala, with production-quality engineering habits.
• Deep understanding of data modeling for event, time-series and entity/state data; experience with stream and batch ingestion frameworks (e.g., Kafka, Flink, Spark).
• Experience running data infrastructure on Kubernetes and/or cloud environments (AWS/Azure/GCP/On-prem).
• Track record of owning systems end-to-end: design, deployment, monitoring and production support.

Preferred Qualifications:

• Prior experience in telecom OSS/BSS, network management systems, or telemetry-heavy domains - ideally exposure to PM, FM, IM or CM data (3GPP counters, SNMP traps, syslog, NETCONF/YANG, topology).
• Hands-on YugabyteDB experience, including geo-partitioning, xCluster/geo-distribution and PostgreSQL wire-compatibility nuances.
• Experience building AI-ready data layers: feature stores, training data extraction, near-real-time serving for ML inference.
• Familiarity with data governance tooling (schema registries, lineage, catalogs) and event-streaming ecosystems.
• Experience with S3-compatible data lakes and lakehouse patterns.
• Contributions to open-source data infrastructure projects.

Our worldwide practices describe specific behaviours that make Rakuten unique and united across the world. We expect Rakuten employees to model these 5 Shugi Principles of Success.

• Always improve, always advance. Only be satisfied with complete success - Kaizen.
• Be passionately professional. Take an uncompromising approach to your work and be determined to be the best.
• Hypothesize - Practice - Validate - Shikumika. Use the Rakuten Cycle to success in unknown territory.
• Maximize Customer Satisfaction. The greatest satisfaction for workers in a service industry is to see their customers smile.
• Speed!! Speed!! Speed!! Always be conscious of time. Take charge, set clear goals, and engage your team.