Python Data Engineer (Blr/Chn/Hyd/Kochi/Kol/Pune)

Tata Consultancy Services; Hyderabad, Pune District, Bengaluru; On-site; INR 1,500,000 - 2,600,000; Full time

Job summary

Tata Consultancy Services in Hyderabad seeks an experienced Python Data Engineer to design, build, and optimize scalable data pipelines across cloud environments. The role requires strong hands-on data engineering skills and collaboration with business and technology stakeholders in a customer-facing setting.

Key responsibilities include developing ETL/ELT pipelines, implementing data quality checks, and working with Azure, AWS, and GCP-based platforms to deliver enterprise-grade data solutions.

Qualifications

• Hands-on Python data processing and transformation experience.
• Expertise in PySpark and distributed data processing.
• Strong SQL skills with complex joins, aggregations and query optimization.
• Experience building ETL/ELT pipelines and data ingestion frameworks.
• Hands-on experience with cloud platforms (Azure, AWS, GCP).
• Strong understanding of Data Warehousing, Dimensional Modeling and Data Modeling Concepts.
• Experience handling structured and semi-structured data.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Python and PySpark.
• Build ETL/ELT frameworks for ingesting, transforming, and processing large datasets.
• Develop data ingestion solutions from multiple sources.
• Implement data transformation, cleansing, validation, and quality checks.
• Work with cloud-based data platforms to build modern data solutions.
• Collaborate with teams to deliver scalable data solutions.
• Design and maintain data models supporting analytics and reporting.
• Optimize pipeline performance, scalability, and reliability.
• Ensure data quality and governance standards are maintained.
• Support Data Warehousing and enterprise analytics initiatives.
• Participate in Agile ceremonies and contribute to continuous improvements.
• Translate business requirements into technical solutions.

Skills

Python

PySpark

SQL

ETL/ELT

Cloud platforms

Data pipelines

Agile

JIRA

Stakeholder management

Job description

We are looking for an experienced Python Data Engineer with strong expertise in Python, PySpark, SQL, ETL/ELT, and Cloud Data Platforms. The ideal candidate will design, build, and optimize scalable data pipelines, support modern data architectures, and deliver enterprise-grade data solutions across cloud environments.

The role requires strong hands‑on data engineering experience, excellent problem‑solving skills, and the ability to work closely with business and technology stakeholders in a customer‑facing environment.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Python and PySpark.
• Build ETL/ELT frameworks for ingesting, transforming, and processing large datasets.
• Develop data ingestion solutions from multiple structured and semi‑structured data sources.
• Implement data transformation, cleansing, validation, and quality checks.
• Work with cloud‑based data platforms to build modern data engineering solutions.
• Collaborate with technology, architecture, product, and design teams to deliver scalable data solutions.
• Design and maintain data models supporting analytics and reporting requirements.
• Optimize data pipeline performance, scalability, and reliability.
• Ensure data quality and governance standards are maintained.
• Support Data Warehousing and enterprise analytics initiatives.
• Participate in Agile ceremonies and contribute to continuous improvement initiatives.
• Work closely with customers and stakeholders to understand business requirements and translate them into technical solutions.

Required Skills

• Strong hands‑on experience with Python for data processing and transformation.
  
• Expertise in PySpark and distributed data processing frameworks.
  
• Strong SQL skills, including:
  
• Complex joins
• Aggregations
• Query optimization
• Data transformation
• Experience building ETL/ELT pipelines and data ingestion frameworks.
  
• Hands‑on experience with cloud platforms:
  
• Azure
• AWS
• GCP
• Strong understanding of:
  
• Data Warehousing
• Dimensional Modeling
• Data Modeling Concepts
• Experience handling structured and semi‑structured data.
  

Preferred Skills

• Azure Data Factory (ADF).
• Databricks.
• Azure Synapse Analytics.
• Kafka.
• Azure Event Hubs.
• CI/CD implementation for Data Engineering workflows.
• Agile methodologies and JIRA.
• BFSI, Insurance, or Investment domain experience.
• LOMA certification or similar industry certifications.

Desired Candidate Profile

• Strong analytical and problem‑solving skills.
• Excellent communication and stakeholder management abilities.
• Strong ownership mindset and accountability.
• Ability to work effectively in customer‑facing environments.
• Experience working in Agile and cross‑functional teams.
• Passion for building scalable and reliable data platforms.