Python with Data Engineer

Tata Consultancy Services; Chennai District, Bengaluru, Hyderabad; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Tata Consultancy Services in Chennai is seeking a Data Engineer to build and support scalable data pipelines and transformation processes using Python, PySpark, SQL, and cloud data platforms.

You will design and maintain ETL/ELT pipelines, process large data volumes, and collaborate with business teams to deliver robust data solutions while ensuring data quality and governance.

Qualifications

• 5+ years of experience in Data Engineering.
• Strong Python for data processing and transformation.
• Experience with PySpark and distributed data processing.
• Strong SQL skills with complex joins and optimizations.
• Experience building ETL/ELT pipelines and data ingestion.
• Hands-on experience with cloud platforms (Azure, AWS, GCP).
• Good understanding of Data Warehousing and dimensional modeling.
• Experience working with structured and semi-structured data.

Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines.
• Process and transform large volumes of structured and semi-structured data.
• Develop scalable solutions using Python, PySpark, and SQL.
• Optimize data workflows, queries, and data processing performance.
• Collaborate with business and technical teams to deliver data solutions.
• Ensure data quality, reliability, and governance standards.

Skills

Data Engineering

Python

PySpark

SQL

ETL/ELT pipelines

Cloud platforms

Data Warehousing

Dimensional Modeling

Structured data handling

Kafka/Streaming

Tools

Azure Databricks

Azure Data Factory

Azure Synapse

AWS

GCP

Kafka

Job description

Job Summary

We are looking for a skilled Data Engineer with experience in Python, PySpark, SQL, and Cloud Data Platforms to build and support scalable data pipelines, data transformation processes, and data engineering solutions.

Key Responsibilities

• Design, develop, and maintain ETL/ELT data pipelines.
• Process and transform large volumes of structured and semi-structured data.
• Develop scalable solutions using Python, PySpark, and SQL.
• Optimize data workflows, queries, and data processing performance.
• Collaborate with business and technical teams to deliver data solutions.
• Ensure data quality, reliability, and governance standards.

Must-Have Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Python for data processing and transformation.
• Experience with PySpark and distributed data processing frameworks.
• Strong proficiency in SQL, including complex joins, aggregations, and query optimization.
• Experience building ETL/ELT pipelines and data ingestion frameworks.
• Hands-on experience with Azure, AWS, or GCP cloud platforms.
• Good understanding of Data Warehousing and Dimensional Modeling concepts.
• Experience working with structured and semi-structured datasets.

Good-to-Have Skills

• Experience with Azure Databricks, Azure Data Factory, or Azure Synapse.
• Exposure to Kafka, Event Hubs, or other streaming technologies.
• Experience with CI/CD pipelines and DevOps practices.

Good-to-Have Skills

• Experience with CI/CD pipelines and DevOps practices.
• Working knowledge of Agile/Scrum methodologies and JIRA.
• BFSI/Investment domain experience or relevant certifications (e.g., LOMA) is an advantage.

Skill

Python, PySpark, SQL, Data Engineer, ETL, ELT, Azure, AWS, GCP, Databricks, Azure Data Factory, Synapse, Data Warehousing, Dimensional Modeling, Kafka, Big Data, Cloud Data Engineering, Data Pipeline, Spark.