AWS Data Engineer - PySpark, ETL & Data Lake Expert

Visa Hunt; Singapore; On-site; SGD 110,000 - 170,000; Full time

Job summary

Visa Hunt in Singapore seeks an experienced data engineer to design and maintain scalable data pipelines using PySpark on AWS. You will work with data lakes, governance, and modernization initiatives, collaborating with business stakeholders to deliver reliable data solutions.

The role emphasizes hands-on development in Python, ETL/ELT patterns, and infrastructure as code with Terraform, alongside CI/CD automation and monitoring to ensure cost-efficient, compliant data pipelines.

Qualifications

• Hands-on PySpark and Python for data engineering.
• ETL pipelines with AWS Glue.
• Workflow orchestration with AWS Step Functions.
• Serverless solutions with AWS Lambda.
• Terraform IaC for cloud infra.
• CI/CD pipelines with GitLab, GitHub Actions or Jenkins.
• Good understanding of AWS services, data lakes, IAM, S3, CloudWatch.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark on AWS.
• Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
• Develop serverless applications and automation using AWS Lambda.
• Write clean, efficient PySpark and Python code following engineering best practices.
• Provision and manage cloud infrastructure using Terraform (IaC).
• Implement and maintain CI/CD pipelines to automate deployment, testing, and infra changes.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost.
• Collaborate with business stakeholders to deliver data solutions.
• Follow DevOps, security, and coding standards throughout the engagement.

Skills

PySpark

Python

AWS Glue

Step Functions

Lambda

Terraform

CI/CD

AWS services

Data lakes

Tools

GitLab

GitHub Actions

Jenkins

Job description

Visa Hunt in Singapore seeks an experienced data engineer to design and maintain scalable data pipelines using PySpark on AWS. You will work with data lakes, governance, and modernization initiatives, collaborating with business stakeholders to deliver reliable data solutions.

The role emphasizes hands-on development in Python, ETL/ELT patterns, and infrastructure as code with Terraform, alongside CI/CD automation and monitoring to ensure cost-efficient, compliant data pipelines.