Senior Data Engineer - AWS Data Lake & Lakehouse

New Era Technology; Kuala Lumpur; On-site; MYR 180,000 - 240,000; Full time

Job summary

New Era Technology is seeking a Senior Data Engineer to be deployed at a client site in Kuala Lumpur. You will design, develop, and maintain scalable data platforms on AWS, focusing on data lakes, data warehouses, and automated pipelines for analytics and reporting.

The role requires hands-on expertise in PySpark, Terraform, and data governance, with responsibilities spanning DR planning, data quality, and collaboration with business stakeholders.

Qualifications

• Bachelor's degree in computer science or related field
• Minimum 3+ years of experience in Data Engineering, Data Warehousing, or related roles
• Strong hands-on experience with SQL, Python, PySpark, Terraform
• Experience with AWS services including AWS Glue, Redshift, S3, RDS, SNS, Step Functions, CloudWatch, IAM, EventBridge, DataZone
• Experience designing and supporting enterprise Data Lake and Data Warehouse solutions
• Experience with Apache Iceberg and large-scale data processing platforms
• Understanding of ETL/ELT methodologies, data modeling, and data integration
• Experience implementing Infrastructure as Code (IaC) using Terraform
• Knowledge of cloud security, IAM permissions, and access control principles
• Experience planning, executing, and supporting Disaster Recovery activities
• Familiarity with AI-assisted software development practices

Responsibilities

• Design, develop, and maintain scalable, secure data pipelines for analytics and reporting
• Build and optimize ETL/ELT processes to ingest, transform, and deliver data to Data Lake and Data Warehouse
• Design and maintain cloud-native data solutions on AWS
• Develop scalable batch and near real-time data processing
• Develop and maintain Apache Iceberg datasets for analytics workloads
• Ensure timely and accurate data loading from source systems into Data Lake
• Monitor and troubleshoot data ingestion and resolve data issues
• Perform root cause analysis and implement sustainable fixes
• Optimize data processing performance, scalability, and cost
• Support enterprise reporting, analytics, and regulatory data needs
• Design, build, and support AWS-based data platforms and IaC via Terraform
• Automate environment provisioning, deployment, and operations
• Implement monitoring, alerting, and operations frameworks for stability

Skills

SQL

Python

PySpark

Terraform

AWS

Data Lake

Data Warehouse

ETL/ELT

Data Modeling

Education

Bachelor's degree in Computer Science / IT / Data Engineering

Tools

AWS Glue

Amazon Redshift

Amazon S3

AWS DataZone

Job description

New Era Technology is seeking a Senior Data Engineer to be deployed at a client site in Kuala Lumpur. You will design, develop, and maintain scalable data platforms on AWS, focusing on data lakes, data warehouses, and automated pipelines for analytics and reporting.

The role requires hands-on expertise in PySpark, Terraform, and data governance, with responsibilities spanning DR planning, data quality, and collaboration with business stakeholders.