AWS Redshift Senior Data Engineer

Tata Consultancy Services; Hyderabad, Chennai District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services is seeking a senior data engineer to design and maintain cloud-based data warehouses and robust ETL/ELT pipelines. You will work with AWS, Redshift, and BigQuery to deliver real-time and batch data solutions.

The role requires strong SQL, Python skills (pandas, numpy, Flask) and the ability to communicate with technical and non-technical stakeholders in an Agile setting.

Qualifications

• 6+ years of experience with AWS (including Redshift) and other cloud technologies.
• 6+ years designing and maintaining data warehouses (Redshift, BigQuery) and ETL/ELT pipelines.
• Experience with real-time and batch data pipelines, including API integrations.
• Experience with data onboarding blueprints mapping source-target data strategies and aggregations.
• Advanced SQL proficiency, including query optimization, indexing, and partitioning.
• Hands-on experience with Python for data processes using pandas, numpy, and Flask.
• Experience with data monitoring, detection, troubleshooting and validation frameworks.
• Proven track record of delivering well-designed, well-tested pipelines in an Agile environment with stakeholders.

Skills

Communication skills

AWS (Redshift)

Cloud technologies

Data warehousing

ETL/ELT pipelines

Redshift

BigQuery

Real-time data pipelines

API integrations

Data onboarding

SQL proficiency

Query optimization

Indexing

Partitioning

Python for data

Flask

Pandas/Numpy

Data monitoring

Validation frameworks

Agile collaboration

Tools

Redshift

BigQuery

Job description

Preferred candidate profile

• Exceptional communication skills and can communicate
• 6+ years of experience with AWS (including Redshift) and other cloud technologies
• 6+ years of experience designing and maintaining data warehouses (e.g., Redshift, Big Query) as well as building complex ETL/ELT pipelines.
• Expertise in working with real-time and batch data pipelines, including API-based integration workflows.
• Experience in maintaining and executing data onboarding blueprints which map source and target data strategies and aggregations.
• Advanced SQL proficiency, including concepts such as query optimization, indexing, and partitioning.
• Hands-on experience with Python for data-related processes, using libraries like pandas, numpy, and frameworks like Flask.
• Experience with data monitoring, detection, troubleshooting as well as validation frameworks.
• Proven track record of delivering well-designed, well-tested and performant software solutions and pipelines within an agile environment.effectively to both technical and non-technical stakeholders.