Senior Data Engineer

Airtel; New Delhi; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Airtel in Delhi is looking for a data engineering specialist to own the technical design and architecture of scalable data-processing solutions on GCP. You will build batch, incremental and streaming pipelines and implement data transformations using BigQuery, SQL and Dataform/dbt.

The role involves creating reusable, metadata-driven frameworks, designing orchestration patterns with Cloud Composer/Airflow, and developing distributed processing solutions with Dataflow/Beam and Spark.

Responsibilities

• Own the technical design and architecture of scalable data-processing and transformation solutions on GCP.
• Design and build production-grade batch, incremental and streaming data pipelines.
• Develop and optimise complex transformations using BigQuery, SQL and Dataform / dbt.
• Build reusable, metadata-driven and configuration-driven frameworks that reduce bespoke pipeline development.
• Design orchestration patterns using Cloud Composer / Apache Airflow, including dynamic workflows, dependencies, retries and recovery.
• Develop distributed and streaming processing solutions using Dataflow / Apache Beam and Dataproc / Spark.
• Define standards for data modelling, schema design, schema evolution and data contracts.
• Design and evolve modern analytical storage patterns, including Apache Iceberg.

Job description

Responsibilities:

• Own the technical design and architecture of scalable data-processing and transformation solutions on GCP.
• Design and build production-grade batch, incremental and streaming data pipelines.
• Develop and optimise complex transformations using BigQuery, SQL and Dataform / dbt.
• Build reusable, metadata-driven and configuration-driven frameworks that reduce bespoke pipeline development.
• Design orchestration patterns using Cloud Composer / Apache Airflow, including dynamic workflows, dependencies, retries and recovery.
• Develop distributed and streaming processing solutions using Dataflow / Apache Beam and Dataproc / Spark.
• Define standards for data modelling, schema design, schema evolution and data contracts.
• Design and evolve modern analytical storage patterns, including Apache Iceberg.