AWS Redshift

Tata Consultancy Services; Hyderabad, Chennai District; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services is seeking a senior data engineer to design and maintain data warehouses, build complex ETL/ELT pipelines, and manage real-time and batch data flows. You will collaborate with stakeholders to map data strategies, optimize SQL, and implement Python-based data processes using Pandas, NumPy, and Flask.

The role requires strong communication, extensive AWS experience, and a track record of delivering performant pipelines in Agile settings.

Qualifications

• Exceptional communication to technical and non-technical stakeholders.
• 7+ years AWS experience including Redshift.
• 7+ years data warehouse design and ETL/ELT pipelines.
• Experience with real-time and batch data pipelines and API integrations.
• Experience with data onboarding blueprints mapping source and target data strategies.
• Advanced SQL including optimization, indexing and partitioning.
• Python for data processes using pandas, numpy, and Flask.
• Experience with data monitoring, troubleshooting and validation frameworks.
• Proven track record delivering well-designed, tested pipelines in Agile environments.

Responsibilities

• Communicate effectively with stakeholders across teams.
• Design and maintain data warehouses (Redshift, BigQuery) and build ETL/ELT pipelines.
• Develop and manage real-time and batch data pipelines, including API-based integrations.
• Maintain data onboarding blueprints mapping source and target data strategies.
• Write advanced SQL with optimization, indexing, and partitioning.
• Develop data processes using Python with pandas, numpy, and Flask.
• Implement data monitoring, detection, troubleshooting and validation workflows.
• Deliver well-designed, tested, and performant software solutions within Agile environments.

Skills

Exceptional communication

AWS (Redshift)

Data warehouse design

ETL/ELT pipelines

Real-time data pipelines

API-based integration

Data onboarding blueprints

Advanced SQL

Python (pandas/numpy)

Agile environment

Tools

Python

Flask

Pandas

NumPy

Job description

Role & responsibilities

• Exceptional communication skills and can communicate effectively to both technical and non-technical stakeholders.
• 7+ years of experience with AWS (including Redshift) and other cloud technologies
• 7+ years of experience designing and maintaining data warehouses (e.g., Redshift, Big Query) as well as building complex ETL/ELT pipelines.
• Expertise in working with real-time and batch data pipelines, including API-based integration workflows.
• Experience in maintaining and executing data onboarding blueprints which map source and target data strategies and aggregations.
• Advanced SQL proficiency, including concepts such as query optimization, indexing, and partitioning.
• Hands-on experience with Python for data-related processes, using libraries like pandas, numpy, and frameworks like Flask.
• Experience with data monitoring, detection, troubleshooting as well as validation frameworks.
• Proven track record of delivering well-designed, well-tested and performant software solutions and pipelines within an agile environment.