Data Engineer — Python / Kafka Streams

Avendumtech; Gurgaon; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

Avendumtech in Gurgaon, India is looking for a skilled engineer to architect streaming telemetry ingestion and manage a multi-vendor network configuration parser pipelines. The role demands expertise in building Python-based ETL pipelines and implementing real-time streaming pipelines using Apache Kafka.

The ideal candidate should have over 2 years of experience with strong SQL knowledge and familiarity with data validation schemas. You will also integrate machine learning models for telemetry classification and optimize big-data processing performance.

Qualifications

• 2+ years of experience in Python engineering with a focus on data pipelines.
• Hands-on experience with Apache Kafka, PySpark, or similar streaming tools.
• Strong SQL skills and experience with time-series or columnar databases.

Responsibilities

• Design and build Python-based ETL pipelines to parse multi-GB RAN configuration dumps.
• Implement real-time streaming pipelines using Apache Kafka and Python.
• Integrate machine learning models for telemetry classification and anomaly detection.
• Maintain data warehouse schemas and optimize big-data processing performance.

Skills

Python engineering

Apache Kafka

SQL

Data pipelines

Data validation schemas

Tools

PySpark

Job description

Architect our streaming telemetry ingestion and multi-vendor network configuration parser pipelines.

Key Responsibilities

• Design and build Python-based ETL pipelines to parse multi-GB RAN configuration dumps.
• Implement real-time streaming pipelines using Apache Kafka and Python.
• Integrate machine learning models for telemetry classification and anomaly detection.
• Maintain data warehouse schemas and optimize big-data processing performance.

What We Look For

• 2+ years of experience in Python engineering with a focus on data pipelines.
• Hands-on experience with Apache Kafka, PySpark, or similar streaming tools.
• Strong SQL skills and experience with time-series or columnar databases.
• Familiarity with data validation schemas (Zod, Pydantic) and parsing libraries.