Data Engineer

ELPA ASIA PTE. LTD.; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A data solutions company located in Singapore seeks a skilled data engineer to enhance and implement data pipelines using AWS managed components. The successful candidate will have over 5 years of experience in data engineering, proficient in Python, SQL, and AWS tools. Responsibilities include monitoring, documenting, and testing data pipelines, ensuring high standards of quality and reliability.

Qualifications

• At least 5 years hands-on working experience in data engineering, designing and building ETL pipelines.
• Experience in project execution and demonstrated technical expertise.
• Proficient in Python, SQL and Shell scripts (Linux).

Responsibilities

• Enhance the existing data pipeline using industry-standard tools.
• Redesign and implement data pipeline to use AWS managed components.
• Implement data pipeline monitoring to alert for issues.
• Investigate and resolve any data pipeline issues.
• Document the data pipeline solution.
• Implement unit testing and integration testing for data pipeline component.

Skills

Data engineering

ETL pipeline design

Python

SQL

Linux Shell scripting

AWS tools

Airflow

Job description

Responsibilities

• Enhance the existing data pipeline using industry-standard tools and mechanisms.
• Redesign and implement data pipeline to use AWS managed components.
• Implement data pipeline monitoring to alert us if there is any issue with the pipeline.
• Investigate and resolve any issue with data pipeline.
• Document the data pipeline solution and how to operate the data pipeline.
• Implement unit testing and integration testing for data pipeline component.

Requirements

• At least 5 years hands-on working experience in data engineering, designing and building ETL pipelines.
• Experience in project execution and demonstrated technical expertise.
• Proficient in Python, SQL and Schell scripts (Linux).
• Airflow or similar tool to create pipelines.
• AWS tools like S3, Glue, AWS EMR, Lambda, Step functions.
• Familiar with tools like AWS Batch, Athena, file formats like Parquet, NoSQL knowledge and tools like AWS OpenSearch.