Senior Data Engineer

Tanla Platforms Limited; Hyderabad; On-site; INR 3,000,000 - 4,200,000; Full time

Job summary

Tanla Platforms Limited is seeking a seasoned Data Engineer to design and build scalable ETL/ELT pipelines for large-scale analytics of telecom datasets, including CDRs. You will architect lakehouse platforms using Iceberg/Delta/Hudi with catalog services and optimize SQL and Spark jobs for real-time insights across the platform ecosystem.

The role requires 7–10 years in data engineering, strong SQL and distributed processing skills, and hands-on cloud experience with AWS/Azure/GCP.

Qualifications

• 7–10 years of experience in Data Engineering or related roles.
• Strong expertise in Advanced SQL with performance tuning and partitioning.
• Hands-on with Spark (PySpark), Hadoop, Kafka and distributed processing.
• Experience with lakehouse tech like Iceberg, Delta Lake or Hudi.
• Knowledge of metadata/catalog systems (Nessie, Glue, Hive Metastore).
• Familiarity with analytical engines (ClickHouse, Dremio, Trino, Presto).
• Proficient in Python; exposure to Scala is a plus.
• Experience with Airflow or Kestra for orchestration; cloud exposure (AWS/Azure/GCP).
• Experience in telecom data systems or CDR processing is preferred.

Responsibilities

• Design and implement scalable ETL/ELT pipelines for analytics and migration workloads.
• Build modern data lakehouse platforms using Iceberg, Delta Lake, or Hudi with catalog services.
• Develop and optimize high-performance SQL queries and distributed data processing jobs.
• Design and manage data warehouses using Snowflake, Redshift, Trino, or similar.
• Build ingestion/transformation pipelines using object storage like S3, ADLS, GCS.
• Process telecom datasets and CDRs efficiently at scale.
• Implement orchestration workflows using Airflow, Kestra, or similar tools.
• Ensure data quality, governance, lineage, observability, scalability and cost optimization.

Skills

Advanced SQL

Apache Spark / PySpark

Hadoop

Kafka

Lakehouse technologies

Python

Scala

Airflow

Cloud platforms (AWS/Azure/GCP)

Data modeling & warehousing

Telecom data / CDR experience

Tools

Nessie

Glue / Hive Metastore

Kafka Connect

Dremio / Snowflake / Redshift familiarity

Job description

We are looking for a highly skilled Data Engineer with strong experience in building scalable ETL/ELT pipelines, distributed data systems, and modern lakehouse architectures. The ideal candidate will work on large-scale telecom and CPaaS datasets, including Call Detail Records (CDR), enabling real-time analytics and business intelligence across the platform ecosystem.

What you’ll be Responsible for?

• Design and implement scalable ETL/ELT pipelines for large-scale analytics, data processing, and migration workloads.
• Build modern data lakehouse platforms using Iceberg, Delta Lake, or Hudi with catalog services like Nessie, AWS Glue, or Hive Metastore.
• Develop and optimize high-performance SQL queries and distributed data processing jobs using Spark (PySpark), Hadoop, and Kafka.
• Design and manage data warehouses and analytical platforms using Snowflake, ClickHouse, Dremio, Redshift, Trino, or Presto.
• Build ingestion and transformation pipelines using object storage systems such as Amazon S3, Azure Data Lake, GCS, or Nutanix Object Storage.
• Process and transform telecom datasets and Call Detail Records (CDR) efficiently at scale.
• Implement orchestration workflows using Airflow, Kestra, or similar workflow engines.
• Ensure data quality, governance, lineage, observability, scalability, and cost optimization across distributed systems.
• Build reusable frameworks for bulk data movement, ingestion acceleration, and transformation at scale.

What you’d have?

• 7 –10 years of experience in Data Engineering or related roles.
• Strong expertise in Advanced SQL including query optimization, partitioning, indexing, and performance tuning.
• Hands‑on experience with Apache Spark (PySpark), Hadoop, Kafka, and distributed data processing systems.
• Strong expertise in lakehouse technologies such as Iceberg, Delta Lake, or Hudi.
• Experience with metadata/catalog systems including Nessie, Glue, or Hive Metastore.
• Knowledge of analytical engines such as ClickHouse, Dremio, Trino, or Presto.
• Strong understanding of Parquet, ORC, and Avro data formats.
• Experience with object storage systems like S3, ADLS, GCS, or Nutanix Object Storage.
• Strong programming skills in Python / PySpark / Scala.
• Experience with Airflow, Kestra, or similar orchestration tools.
• Hands‑on exposure to AWS, Azure, or GCP cloud platforms.
• Experience in telecom data systems or CDR processing is highly preferred.
• Impactful Work: Build large-scale data platforms and analytics systems that power real-time communication products used by millions globally.
• Tremendous Growth Opportunities: Accelerate your career by solving complex engineering challenges in a fast‑growing CPaaS and product-driven environment.
• Innovative Environment: Work alongside world‑class engineers building cutting-edge distributed data systems, lakehouse architectures, and cloud‑native platforms.

Tanla is an equal opportunity employer. We champion diversity and are committed to creating an inclusive environment for all employees.