AWS Databricks

Tata Consultancy Services; Hyderabad, Chennai District, Bengaluru; On-site; INR 3,000,000 - 5,500,000; Full time

Job summary

Tata Consultancy Services in Hyderabad, Bengaluru and Chennai is seeking an experienced AWS Databricks data engineer to design and optimize scalable data pipelines on AWS. You will work with PySpark, Python, SQL, and Delta Lake to deliver robust ETL/ELT solutions, build Data Lake/Lakehouse architectures, and ensure data quality and governance across projects.

Collaborate with architects and stakeholders to translate requirements into reliable data solutions, mentor junior engineers, and support

Qualifications

• 8–12 years of data engineering experience.
• Strong hands-on experience with AWS Glue and Databricks.
• Proficient in PySpark, Python, and SQL.
• Experience designing and building ETL/ELT pipelines and data lakehouse architectures.
• Ability to collaborate with stakeholders and mentor juniors.

Responsibilities

• Design, build, and maintain scalable data pipelines using AWS Glue and Databricks.
• Develop and optimize ETL/ELT workflows using PySpark and Python.
• Ingest structured and unstructured data; implement data quality checks.
• Build and manage Data Lake/Lakehouse on AWS; monitor pipelines.
• Collaborate with architects and teams; ensure security and governance.
• Mentor junior data engineers and support production deployments.

Skills

AWS Glue

Databricks

PySpark

Python

SQL

Data Warehousing

ETL/ELT

Data Lake/Lakehouse

GitHub

Spark tuning

Tools

Delta Lake

GitHub

Job description

Virtual drive 18th september 2026

Role: AWS Databricks

Experience: 8 - 12 yrs

location: Chennai, Bangalore, Hyderabad

Responsibility:

• Design, build, and maintain scalable data pipelines using AWS Glue and Databricks.
• Develop and optimize ETL/ELT workflows using PySpark and Python.

• Implement data ingestion frameworks for structured and unstructured data.
• Build and manage Data Lake/Lakehouse solutions on AWS.
• Create reusable data processing frameworks and data quality checks.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Collaborate with business stakeholders, architects, and development teams to gather requirements and deliver solutions.
• Ensure adherence to security, governance, and data management best practices.
• Participate in code reviews and mentor junior data engineers.
• Support production deployments and resolve critical issues.

Skills:

• Strong hands-on experience in AWS Glue
• Expertise in Databricks (Notebooks, Jobs, Delta Lake)
• Advanced programming skills in PySpark and Python
• Strong SQL and Data Warehousing concepts
• Experience with AWS services such as S3, EMR, Lambda, IAM, Athena, Redshift
• Experience in designing and developing ETL/ELT pipelines
• Data Lake and Lakehouse architecture implementation
• Git/GitHub for version control
• Performance tuning and optimization of Spark jobs