Sr Data Engineer

Asian Technology Solutions; Kuala Lumpur; On-site; MYR 180,000 - 270,000; Full time

Job summary

Asian Technology Solutions in Kuala Lumpur is seeking an experienced AWS Data Engineer to collaborate with business and data stakeholders, translating requirements into scalable data pipelines and ETL designs.

You will design and develop robust ETL pipelines with AWS Glue PySpark on S3, implement complex transformations, and contribute to governance, security, and DevOps practices.

Qualifications

• 7–10 years of AWS Data Engineering experience.
• Strong experience with AWS Glue PySpark and data processing on S3.
• Proven expertise in ETL development and complex data transformations.
• Hands-on experience with AWS Step Functions, EventBridge, and orchestration patterns.
• Proficiency in Terraform Infrastructure as Code.
• Strong SQL and data modeling skills.
• Experience with data validation, reconciliation, and quality frameworks.
• Solid understanding of IAM, security, and cloud best practices.

Responsibilities

• Collaborate with business and data stakeholders to understand data requirements, transformation rules, and use cases.
• Translate functional requirements into scalable data engineering and ETL design solutions.
• Design and develop robust ETL pipelines using AWS Glue PySpark.
• Implement complex data transformations including joins, aggregations, conditional logic, and business rule processing.
• Build reusable, metadata-driven ETL frameworks where applicable.
• Optimize data pipelines for performance, scalability, and reliability.
• Perform unit testing and end-to-end validation of data pipelines.
• Implement data quality checks including integrity, completeness, and reconciliation.
• Validate transformation logic and ensure accuracy of aggregated and processed data.
• Develop and manage workflow orchestration using AWS Step Functions and EventBridge.
• Implement security governance with IAM and Lake Formation; monitor, log, and troubleshoot pipelines.

Skills

AWS Glue PySpark

S3 data processing

ETL development

AWS Step Functions

AWS EventBridge

Terraform IaC

SQL

Data validation & quality

IAM & security

Tools

Terraform

DynamoDB

AWS Lake Formation

Job description

Jora Malaysia will close on 9th September 2026. Thank you for being with us, we are cheering you on as you continue your career journey.

Asian Technology Solutions – Kuala Lumpur, Kuala Lumpur

Collaborate with business and data stakeholders to understand data requirements, transformation rules, and use casesTranslate functional requirements into scalable data engineering and ETL design solutionsDefine approach for data ingestion, transformation, orchestration, and access control

Design and develop robust ETL pipelines using AWS Glue PySparkImplement complex data transformations including joins, aggregations, conditional logic, and business rule processing

Build reusable, metadata-driven ETL frameworks where applicable

Optimize data pipelines for performance, scalability, and reliability

Perform unit testing and end-to-end validation of data pipelines

Implement data quality checks including integrity, completeness, and reconciliation

Validate transformation logic and ensure accuracy of aggregated and processed data

3. Orchestration IntegrationDevelop and manage workflow orchestration using AWS Step Functions and EventBridgeImplement event-driven and schedule-based pipeline execution strategiesEnsure seamless integration between data pipelines, control frameworks, and downstream systems

4. Infrastructure Deployment IaCProvision and manage cloud infrastructure using Terraform Infrastructure as CodeDeploy and configure AWS services including Glue, Lambda, DynamoDB, and orchestration componentsEnsure consistent, repeatable, and scalable deployments aligned with DevOps practices

6. Security GovernanceImplement secure data access controls using IAM and Lake FormationEnsure compliance with data governance policies and role-based access requirementsManage data security, encryption, and access auditing

Set up monitoring, logging, and alerting mechanisms for data pipelines e.g., SNS, audit logs

Troubleshoot data and pipeline issues, ensuring timely resolution

Continuously enhance pipeline performance, reliability, and maintainability

Collaborate in an agile setup to drive continuous delivery and improvements

Must-Have Skills

• 7-10 years of AWS Data Engineering experience.
• Strong experience in AWS Glue PySpark and data processing on S3
• Proven expertise in ETL development and complex data transformations
• Hands-on experience with AWS Step Functions, EventBridge, and orchestration patterns
• Proficiency in Terraform Infrastructure as Code
• Strong SQL and data modeling skills
• Experience with data validation, reconciliation, and quality frameworks
• Solid understanding of IAM, security, and cloud best practices

Nice-to-Have Skills

• Familiarity with Lake Formation and data governance frameworks
• Experience with DynamoDB or metadata-driven ETL architectures
• Exposure to event-driven architectures