Data Engineer Python Kafka

Tata Consultancy Services; Hyderabad, Pune District, Chennai District; On-site; INR 2,000,000 - 4,200,000; Full time

Job summary

Tata Consultancy Services is seeking a Data Engineer focused on Python and Kafka to design, develop and optimize real-time data pipelines across large-scale environments. You will work across data ingestion, processing and integration layers using Python, SQL and Kafka streams.

The role requires hands-on experience with Spark/PySpark, AWS or Azure cloud platforms, and familiarity with CI/CD, Docker and Git.

Qualifications

• Strong experience in Python development and data engineering.
• Hands-on Kafka, Python, SQL and real-time streaming pipelines.
• Experience with ETL/ELT and data integration.
• Knowledge of Spark/PySpark and distributed data processing.
• Exposure to AWS/Azure cloud platforms.
• Understanding of CI/CD, Git, Docker and Agile methodologies.
• Strong problem-solving and analytical skills.

Skills

Python Development

Apache Kafka

SQL

Real-time data streaming

ETL/ELT

Kafka Streams

Spark/PySpark

AWS

Azure

CI/CD

Git

Docker

Linux

Airflow

Tools

Databricks

Snowflake

Azure Data Factory

Airflow

Job description

TCS Hiring | Data Engineer - Python & Kafka

Location: Chennai | Bengaluru | Hyderabad | Pune | Kochi | Kolkata | NCR | Delhi
Experience: 6-10 Years
Date of Interview: 04-Sep-2026
Mode of Interview: Virtual

Required Skills

• Strong experience in Python Development and Data Engineering
• Hands-on expertise in Apache Kafka, Python, and SQL
• Experience building and maintaining real-time data streaming pipelines
• Strong knowledge of ETL/ELT processes and data integration
• Experience with Kafka Producers, Consumers, Topics, Partitions, and Kafka Streams
• Knowledge of Spark/PySpark and distributed data processing frameworks
• Exposure to AWS/Azure cloud platforms
• Understanding of CI/CD, Git, Docker, and Agile methodologies
• Strong problem-solving and analytical skills

Technical Skills

Python, Apache Kafka, SQL, PySpark, Spark, ETL, Data Engineering, Data Pipelines, AWS, Azure, Docker, Git, CI/CD, Linux, Airflow

Preferred Skills

• Experience with real-time streaming and event-driven architectures
• Knowledge of Databricks, Snowflake, or Azure Data Factory
• Exposure to Kafka performance tuning and monitoring
• Experience working with large-scale data processing environments