AWS Data Engineer

Trantor; Dadri; On-site; INR 800,000 - 2,000,000; Full time

Job summary

Trantor in India is seeking a Data Engineer with strong AWS data services to design scalable data pipelines and migration solutions across distributed environments.

You will build cloud-native data engineering solutions on AWS, manage data migrations from legacy systems to AWS S3/Redshift, and implement secure, efficient pipelines with Airflow or Step Functions.

Qualifications

• 5+ years of experience in Data Engineering or Data Platform development.
• Hands-on experience with AWS including Glue, S3, and Lambda.
• Experience with data migrations from legacy data warehouse technologies.
• Proficient in Python and SQL for building scalable data pipelines.
• Strong understanding of ETL/ELT concepts, data partitioning, and distributed processing.
• Experience working with version control systems such as GitLab or Bitbucket.

Responsibilities

• Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3.
• Build and maintain high-performance data ingestion frameworks for processing large-scale datasets.
• Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift.
• Optimize storage and querying strategies using AWS S3 data lakes.
• Develop and maintain data workflow orchestration frameworks using Apache Airflow or AWS Step Functions.
• Automate complex workflows including data ingestion, transformation, validation, and loading processes.
• Lead data migrations from legacy data warehouse technologies to modern AWS data platforms.

Skills

5+ years experience in Data Eng/Data平台

AWS

Python

SQL

ETL/ELT concepts

Data pipelines

GitLab/Bitbucket

Tools

AWS Glue

AWS S3

AWS Lambda

Apache Airflow

AWS Step Functions

AWS Redshift

Job description

We are looking for a highly skilled and motivated Data Engineer with strong expertise in AWS data services to join our data platform team. The ideal candidate will have hands-on experience designing scalable data pipelines, workflow orchestration frameworks, and large-scale data migration solutions.

This role will be responsible for building robust cloud-native data engineering solutions on AWS, migrating datasets from legacy systems and data warehouses, and ensuring secure and efficient data processing pipelines across distributed environments.

Key Responsibilities

• Design and implement scalable ETL/ELT data pipelines using AWS Glue, AWS Lambda, and AWS S3.
• Build and maintain high-performance data ingestion frameworks for processing large-scale datasets.
• Implement data pipelines for data warehousing and analytics platforms such as AWS Redshift.
• Optimize storage and querying strategies using AWS S3 data lakes.

Data Workflow Orchestration

• Develop and maintain data workflow orchestration frameworks using tools such as Apache Airflow or AWS Step Functions.
• Automate complex workflows including data ingestion, transformation, validation, and loading processes.
• Build reusable and configurable workflows to support multiple data processing use cases.

Data Migration & Integration

• Lead data migrations from legacy data warehouse technologies to modern AWS data platforms.
• Perform data migration from RDBMS systems (e.g., MySQL, SQL Server, Oracle) to AWS S3 or AWS Redshift.
• Design scalable migration frameworks for large datasets with minimal downtime.
• Integrate data sources from enterprise applications and external systems.

Data Security & Governance

• Implement secure data pipelines using AWS security best practices.
• Manage access control and data governance using AWS IAM and Lake Formation.
• Ensure data encryption, access management, and compliance across all data platforms.

Performance Optimization & Monitoring

• Monitor data pipelines and troubleshoot performance issues.
• Optimize ETL workflows for scalability, reliability, and cost efficiency.
• Implement logging, monitoring, and alerting mechanisms for data pipelines.

Required Skills & Qualifications (Must Have)

• 5+ years of experience in Data Engineering or Data Platform development
• Strong hands-on experience with:
• AWS
• AWS Glue
• AWS S3
• AWS Lambda
• Experience with Data Workflow Orchestration tools such as Apache Airflow or AWS Step Functions
• Experience performing data migrations from other data warehouse technologies
• Experience performing data migrations from RDBMS systems to AWS S3 or AWS Redshift
• Strong expertise in Python and SQL for building scalable data pipelines
• Solid understanding of ETL/ELT concepts, data partitioning, and distributed data processing
• Experience working with version control systems such as GitLab or Bitbucket
• Strong debugging, analytical thinking, and problem-solving skills
• Basic understanding of Object-Oriented Programming concepts

Industry Knowledge & Experience

• Experience building cloud-native data engineering solutions on AWS
• Experience with data warehouse architectures and large-scale analytics platforms
• Hands-on experience with data extraction, transformation, and migration frameworks
• Experience working in high-volume data environments such as FinTech, analytics platforms, or enterprise data systems

Good to Have Skills

• IBM Cognos
• AWS Lake Formation
• AWS Redshift
• AWS Glue Data Catalog
• AWS SageMaker
• AWS IAM

Soft Skills

• Strong communication skills to present technical solutions and recommendations to stakeholders
• Ability to work cross-functionally in a fast-paced and evolving environment
• Detail-oriented with a proactive approach to identifying and solving data platform challenges
• Ability to collaborate effectively with data scientists, analysts, and platform engineering teams