Data Architect

airtel; Gurugram District; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

Airtel is seeking a mid-senior level Big Data Engineer to enhance its data integration capabilities in Gurugram. In this full-time role, you will leverage your skills in Apache Spark, Python, and SQL to design scalable data workflows and collaborate with cross-functional teams to derive actionable insights.

Qualifications

• 4+ years of experience with Big Data.
• Proficient in Apache Spark, Hive, and Hadoop ecosystem.
• Experience building reusable Spark components.

Responsibilities

• Integrate data from multiple sources into big data environments.
• Design batch and real-time data pipelines.
• Troubleshoot Spark job failures.

Skills

Apache Spark

Python

Pyspark

SQL

Data ingestion

Problem-solving

Tools

Apache Flink

Git

Job description

Get AI-powered advice on this job and more exclusive features.

As a Big Data Engineer, you will play a critical role in integrating multiple data sources, designing scalable data workflows, and collaborating with data architects, scientists, and analysts to develop innovative solutions. You will work with rapidly evolving technologies to achieve strategic business goals.

Must-Have Skills:

• 4+ years of mandatory experience with Big Data.
• 4+ years of mandatory experience in Apache Spark.
• Proficiency in Apache Spark, Hive on Tez, and Hadoop ecosystem components.
• Strong coding skills in Python & Pyspark.
• Experience building reusable components or frameworks using Spark.
• Expertise in data ingestion from multiple sources using APIs, HDFS, and NiFi.
• Solid experience working with structured, unstructured, and semi-structured data formats (Text, JSON, Avro, Parquet, ORC, etc.).
• Experience with UNIX Bash scripting and databases like Postgres, MySQL, and Oracle.
• Ability to design, develop, and evolve fault-tolerant distributed systems.
• Strong SQL skills, with expertise in Hive, Impala, Mongo, and NoSQL databases.
• Hands-on experience with Git and CI/CD tools.
• Experience with streaming data technologies (Kafka, Spark Streaming, Apache Flink, etc.).
• Proficient with HDFS or similar data lake technologies.
• Excellent problem-solving skills — evaluated through coding rounds.

Key Responsibilities:

• Handling existing or new Apache HDFS clusters, including name node, data node, and edge node commissioning & decommissioning.
• Work closely with data architects and analysts to design technical solutions.
• Integrate and ingest data from multiple source systems into big data environments.
• Develop end-to-end data transformations and workflows, ensuring logging and recovery mechanisms.
• Troubleshoot Spark job failures.
• Design and implement batch, real-time, and near-real-time data pipelines.
• Optimize Big Data transformations using Apache Spark, Hive, and Tez.
• Collaborate with Data Science teams to enhance actionable insights.
• Ensure seamless data integration and transformation across multiple systems.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Industries

• Telecommunications and Computer and Network Security

This job posting is active. Referrals increase your chances of interviewing at Airtel by 2x.