AWS Data Engineer (Python & Redshift)_6+yrs

Zorba AI; Hyderabad; On-site; INR 1,200,000 - 2,600,000; Full time

Job summary

A leading technology firm is seeking a Mid-Senior level Data Engineer proficient in developing and orchestrating cloud-native ETL pipelines using Python and AWS services. This full-time role involves optimizing Amazon Redshift database performance, migrating data from legacy systems, and ensuring the scalability of data infrastructure. The ideal candidate has at least 3 years of experience with Redshift and AWS, a degree in Computer Science or a related field, and strong skills in SQL and Python automation.

Qualifications

• 3+ years of hands-on experience with Amazon Redshift, including table design and query optimization.
• Strong proficiency using Git for version control.
• Experience migrating data from legacy systems to cloud data warehouses.
• Strong experience with AWS ecosystem: S3, EC2, IAM, Glue, SSM, and related services.
• Strong proficiency using Git for version control and collaboration on code base.
• Proficient in Python for ETL, automation, and orchestration (including boto3, PySpark).
• Solid background in SQL, with expertise in writing, analyzing, and tuning complex queries for large datasets.
• Experience migrating data from legacy systems (e.g., DB2) to Redshift or other cloud data warehouses.

Responsibilities

• Design, develop, and orchestrate cloud-native ETL pipelines using Python and AWS services.
• Migrate data from legacy systems to Amazon Redshift.
• Optimize Redshift database performance and automate data migration.
• Implement robust logging, monitoring, and error handling in accordance with architecture standards.
• Automate data migration, transformation, and validation processes to support scalable and reliable data workflows.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Maintain and enhance data infrastructure to ensure high availability, scalability, and security.
• Troubleshoot and resolve issues related to data pipelines, Redshift performance, and AWS infrastructure.
• Document data flows, pipeline designs, and operational procedures for ongoing support and knowledge sharing.
• Stay current with AWS best practices and emerging technologies to continuously improve data engineering processes.

Skills

Amazon Redshift

Python

AWS services (S3, EC2, Glue, SSM)

SQL

Git

Git

ETL orchestration

Terraform

Education

Bachelor’s degree in Computer Science or related field

B.Tech/B.E.

Tools

boto3

PySpark

Terraform

Glue

SSM

Redshift

Terraform

Job description

4 days ago Be among the first 25 applicants

This range is provided by Zorba AI. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

Job Summary

The ideal candidate will have experience developing and orchestrating cloud-native ETL pipelines using Python and AWS services (Redshift, EC2, S3, Glue, SSM), including robust logging, error handling, and automation of data migration tasks. They will also possess a strong background and deep understanding of AWS data architecture systems. This role involves designing, implementing, and testing robust data pipelines, optimizing Redshift database performance, and ensuring the scalability and reliability of our data infrastructure.

Key Responsibilities

• Design, develop, and orchestrate cloud-native ETL pipelines using Python and AWS services (Redshift, EC2, S3, Glue, SSM).
• Migrate data from legacy systems (e.g., DB2) to Amazon Redshift, ensuring data quality, integrity, and completeness.
• Optimize Redshift database performance through effective use of sort keys, distribution keys, and query tuning.
• Implement robust logging, monitoring, and error handling in accordance with architecture standards.
• Automate data migration, transformation, and validation processes to support scalable and reliable data workflows.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions that meet business needs.
• Maintain and enhance data infrastructure to ensure high availability, scalability, and security.
• Troubleshoot and resolve issues related to data pipelines, Redshift performance, and AWS infrastructure.
• Document data flows, pipeline designs, and operational procedures for ongoing support and knowledge sharing.
• Stay current with AWS best practices and emerging technologies to continuously improve data engineering processes.

Qualifications

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field.
• 3+ years of hands‑on experience with Amazon Redshift, including table design, data loading, and query optimization.
• Strong experience with AWS ecosystem: S3, EC2, IAM, Glue, SSM, and related services.
• Strong proficiency using Git for version control and collaboration on code base.
• Proficient in Python for ETL, automation, and orchestration (including boto3, PySpark).
• Solid background in SQL, with expertise in writing, analyzing, and tuning complex queries for large datasets.
• Experience migrating data from legacy systems (e.g., DB2) to Redshift or other cloud data warehouses.

Desired Skills

• Deep understanding of Redshift architecture (columnar storage, MPP, node types, concurrency scaling).
• Expertise in designing tables with optimal sort keys, distribution keys, and column encoding.
• Ability to use EXPLAIN plans to diagnose and resolve query performance issues.
• Experience with Redshift‑specific maintenance (VACUUM, ANALYZE, WLM configuration).
• Experience automating performance monitoring, alerting, and cost optimization in Redshift.
• Strong troubleshooting skills for slow queries, data skew, and resource contention.
• Experience with CI/CD for data pipelines and infrastructure‑as‑code (Terraform) is a plus.
• Knowledge of serverless architecture and services (e.g. AWS Lambda).
• Excellent communication skills for collaborating with cross‑functional teams and documenting solutions.

Seniority level Mid‑Senior level

Employment type Full‑time

Job function Information Technology

Industries IT Services and IT Consulting

Referrals increase your chances of interviewing at Zorba AI by 2x