AWS Data Engineer

RAPSYS TECHNOLOGIES PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

RAPSYS TECHNOLOGIES PTE. LTD. is seeking an experienced data engineer to design and maintain scalable data pipelines on AWS, using PySpark and Python.

You will build ETL/ELT workflows with AWS Glue and Step Functions, develop serverless automation with Lambda, and manage infrastructure with Terraform. You will implement CI/CD pipelines, monitor pipelines with CloudWatch, and ensure performance, cost efficiency, and security.

Qualifications

• Hands-on experience with PySpark and Python for data engineering.
• Experience developing ETL pipelines using AWS Glue.
• Proficiency with AWS Step Functions for workflow orchestration.
• Serverless solutions with AWS Lambda.
• Terraform IaC experience.
• CI/CD pipelines using GitLab, GitHub Actions, Jenkins or similar.
• Knowledge of AWS services, data lakes, IAM, S3, CloudWatch and monitoring.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark on AWS.
• Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
• Develop serverless applications and automation using AWS Lambda.
• Write clean, efficient Python/PySpark code following engineering best practices.
• Provision and manage cloud infrastructure using Terraform (IaC).
• Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
• Collaborate with business stakeholders to deliver data solutions.
• Follow DevOps, security, and coding standards throughout the engagement.

Skills

PySpark

Python

Tools

AWS Glue

AWS Step Functions

AWS Lambda

Terraform

CI/CD pipelines tooling

GitLab/GitHub Actions/Jenkins

CloudWatch

IAM

S3

Data Lakes

Job description

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark on AWS.
• Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
• Develop serverless applications and automation using AWS Lambda.
• Write clean, efficient, and maintainable Python/PySpark code following engineering best practices.
• Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
• Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
• Collaborate with business stakeholders to deliver data solutions.
• Follow DevOps, security, and coding standards throughout theengagement.

Required Skills

• Strong hands-on experience with PySpark and Python fordata engineering.
• Experience developing ETL pipelines using AWS Glue.
• Proficiency with AWS Step Functions for workfloworchestration.
• Experience building serverless solutions using AWS Lambda.
• Hands-on experience with Terraform for Infrastructure as Code (IaC).
• Experience implementing CI/CD pipelines using tools such as GitLab, GitHub Actions, Jenkins, or similar.
• Good understanding of AWS services, data lakes, IAM, S3, CloudWatch, and monitoring.