Python with SQL Data Engineer (6 years To 12 years)

Tata Consultancy Services; Hyderabad, Chennai District, Bengaluru; On-site; INR 900,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services in Hyderabad is seeking an experienced data engineer to design and build scalable data pipelines using Python, PySpark, and SQL across multiple data sources.

You will work with cloud data platforms and implement robust ETL/ELT processes, focusing on data quality, performance, and reliability within a collaborative, customer-facing environment.

Qualifications

• Hands-on Python for data processing and transformation.
• Experience with PySpark and distributed data processing frameworks.
• Strong SQL skills with complex joins and aggregations.
• Experience in ETL/ELT pipelines and data ingestion frameworks.
• Hands-on with cloud data platforms (Azure / AWS / GCP).
• Knowledge of data warehousing concepts and dimensional modelling.
• Experience with structured and semistructured data.

Responsibilities

• Collaborate with technology, architecture, product, and design teams to deliver scalable data solutions.
• Design, develop, and optimize data pipelines using Python and PySpark.
• Implement data ingestion, transformation, and validation from multiple sources.
• Ensure data quality, performance optimization, and reliability of pipelines.
• Demonstrate strong communication skills, ownership mindset, and ability to work in a customerfacing environment.

Skills

Python

PySpark

SQL

ETL/ELT pipelines

Cloud data platforms

Data warehousing concepts

Structured data / Semistructured data

Tools

Azure Data Factory / Databricks / Synapse

Kafka / Event Hubs

CI/CD pipelines

JIRA

Job description

Must Have Skills:

• Strong handson experience in Python for data processing and transformation


• Experience with PySpark and distributed data processing frameworks


• Strong proficiency in SQL, including complex joins and aggregations


• Experience in ETL/ELT pipelines and data ingestion frameworks


• Handson experience with Cloud data platforms (Azure / AWS / GCP)


• Knowledge of data warehousing concepts and dimensional modeling


• Experience working with structured and semistructured data

Good to Have Skills:

• Experience with Azure Data Factory / Databricks / Synapse or equivalent


• Exposure to streaming technologies (Kafka, Event Hubs, etc.)


• Experience with CI/CD pipelines for data engineering workflows


• Experience in Agile methodology using JIRA


• Knowledge of BFSI / Investment domain or relevant certifications (e.g., LOMA)

Responsibilities or Expectations from the Role:

• Collaborate with technology, architecture, product, and design teams to deliver scalable data solutions


• Design, develop, and optimize data pipelines using Python and PySpark


• Implement data ingestion, transformation, and validation from multiple sources


• Ensure data quality, performance optimization, and reliability of pipelines


• Demonstrate strong communication skills, ownership mindset, and ability to work in a customerfacing environment