AWS Data Pipeline Engineer — ETL & Data Quality

OnebyZero .; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

OneByZero, a Frontier Systems Integrator in Singapore, seeks a Data Pipeline Engineer to design, build, and optimize scalable data ingestion and transformation pipelines on AWS.

You will implement data quality frameworks, govern data lake environments, and collaborate with cross-functional teams across APAC to deliver enterprise-grade data solutions.

The role emphasizes IaC, CI/CD, testing, and governance, with opportunities to work on mission-critical banking, telco, insurer, and retail projects.

Qualifications

• 5+ years of experience in data engineering or data platform development.
• Hands-on experience with AWS Glue, Lake Formation, S3 and Athena.
• Proficiency in Python, PySpark, and SQL.
• Experience building ETL/ELT pipelines in cloud-native environments.
• IaC experience with Terraform or AWS CDK.
• Strong data modeling, governance, and metadata management.
• Experience with automated testing and deployment practices.

Responsibilities

• Develop end-to-end data ingestion pipelines from source systems through AWS S3, Glue ETL, and Lake Formation curated zones.
• Design and implement ETL transformations based on data mappings.
• Build and maintain data quality validation frameworks for completeness and schema conformance.
• Configure error handling, exception management, and dead-letter queue patterns for failed ingestion records.
• Register and manage datasets within AWS Glue Data Catalog with governance tagging.
• Develop and maintain IaC using Terraform or AWS CDK for data platform resources.
• Execute unit and integration testing to validate pipeline functionality and data accuracy.
• Support UAT by validating output datasets, schemas, and record counts with stakeholders.
• Document pipelines, runbooks, and data flow diagrams.
• Participate in Agile ceremonies including sprint planning, stand-ups, demos, and code reviews.

Skills

Data engineering

ETL development

Python

PySpark

SQL

Data governance

Education

Bachelor's degree in Computer Science, IT, or Engineering

Tools

AWS Glue

AWS Lake Formation

Amazon S3

Amazon Athena

Terraform

AWS CDK

Great Expectations

Job description

OneByZero, a Frontier Systems Integrator in Singapore, seeks a Data Pipeline Engineer to design, build, and optimize scalable data ingestion and transformation pipelines on AWS.

You will implement data quality frameworks, govern data lake environments, and collaborate with cross-functional teams across APAC to deliver enterprise-grade data solutions.

The role emphasizes IaC, CI/CD, testing, and governance, with opportunities to work on mission-critical banking, telco, insurer, and retail projects.