Aws Data Engineer

Trantor; Chandigarh; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Trantor is seeking an AWS Data Engineer to design and implement scalable data pipelines on AWS, migrating data from legacy systems and enabling secure processing across distributed environments.

You will build ingestion frameworks, orchestrate workflows with Airflow or Step Functions, and optimize storage and analytics platforms using S3 data lakes and Redshift.

Qualifications

• 5+ years of experience in Data Engineering or Data Platform development.
• Strong hands-on experience with AWS data services (Glue, S3, Lambda).
• Experience with data workflow orchestration (Airflow or Step Functions).
• Experience performing data migrations from legacy warehouses to AWS.
• Strong Python and SQL skills for building data pipelines.
• Knowledge of ETL/ELT concepts, data partitioning and distributed processing.
• Version control experience (GitLab or Bitbucket).
• Excellent debugging, analytical thinking and problem-solving skills.
• Basic understanding of OOP concepts.

Responsibilities

• Design and implement scalable ETL/ELT data pipelines on AWS (Glue, Lambda, S3).
• Build high-performance data ingestion frameworks for large datasets.
• Develop pipelines for data warehousing and analytics on AWS Redshift.
• Optimize storage and querying using S3 data lakes.
• Develop and maintain workflow orchestration using Airflow or Step Functions.
• Automate complex ingestion, transformation, validation, and loading processes.
• Create reusable and configurable workflows for multiple use cases.
• Lead migrations from legacy systems to AWS data platforms.
• Ensure data security, governance, and encryption across pipelines.

Skills

AWS

AWS Glue

AWS S3

AWS Lambda

Python

SQL

ETL/ELT

Apache Airflow

GitLab

Bitbucket

Data Warehousing

Data Migration

Security & Governance

Tools

Apache Airflow

AWS Step Functions

Job description

We are looking for a highly skilled and motivated AWS Data Engineer with strong expertise in AWS data services to join our data platform team. The ideal candidate will have hands‑on experience designing scalable data pipelines, workflow orchestration frameworks, and large‑scale data migration solutions.

This role will be responsible for building robust cloud-native data engineering solutions on AWS, migrating datasets from legacy systems and data warehouses, and ensuring secure and efficient data processing pipelines across distributed environments.

Key Responsibilities

AWS Data Pipeline Development

• Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3.
• Build and maintain high-performance data ingestion frameworks for processing large-scale datasets.
• Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift.
• Optimize storage and querying strategies using AWS S3 data lakes.

Data Workflow Orchestration

• Develop and maintain data workflow orchestration frameworks using tools such as Apache Airflow or AWS Step Functions.
• Automate complex workflows including data ingestion, transformation, validation, and loading processes.
• Build reusable and configurable workflows to support multiple data processing use cases.

Data Migration & Integration

• Lead data migrations from legacy data warehouse technologies to modern AWS data platforms.
• Perform data migration from RDBMS systems (e.g., MySQL, SQL Server, Oracle) to AWS S3 or AWS Redshift.
• Design scalable migration frameworks for large datasets with minimal downtime.
• Integrate data sources from enterprise applications and external systems.

Data Security & Governance

• Implement secure data pipelines using AWS security best practices.
• Manage access control and data governance using AWS IAM and Lake Formation.
• Ensure data encryption, access management, and compliance across all data platforms.

Performance Optimization & Monitoring

• Monitor data pipelines and troubleshoot performance issues.
• Optimize ETL workflows for scalability, reliability, and cost efficiency.
• Implement logging, monitoring, and alerting mechanisms for data pipelines.

Required Skills & Qualifications (Must Have)

• 5+ years of experience in Data Engineering or Data Platform development
• Strong hands‑on experience with:
• AWS
• AWS Glue
• AWS S3
• AWS Lambda
• Experience with Data Workflow Orchestration tools such as Apache Airflow or AWS Step Functions
• Experience performing data migrations from other data warehouse technologies
• Experience performing data migrations from RDBMS systems to AWS S3 or AWS Redshift
• Strong expertise in Python and SQL for building scalable data pipelines
• Solid understanding of ETL/ELT concepts, data partitioning, and distributed data processing
• Experience working with version control systems such as GitLab or Bitbucket
• Strong debugging, analytical thinking, and problem‑solving skills
• Basic understanding of Object-Oriented Programming concepts

Industry Knowledge & Experience

• Experience building cloud-native data engineering solutions on AWS
• Experience with data warehouse architectures and large-scale analytics platforms
• Hands‑on experience with data extraction, transformation, and migration frameworks
• Experience working in high-volume data environments such as FinTech, analytics platforms, or enterprise data systems

Good to Have Skills

• IBM Cognos
• AWS Athena
• AWS Lake Formation
• AWS Redshift
• AWS Glue Data Catalog
• AWS SageMaker
• AWS IAM

Soft Skills

• Strong communication skills to present technical solutions and recommendations to stakeholders
• Ability to work cross‑functionally in a fast‑paced and evolving environment
• Detail‑oriented with a proactive approach to identifying and solving data platform challenges
• Ability to collaborate effectively with data scientists, analysts, and platform engineering teams