Data Engineer

Espire Infolabs; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

A technology solutions company in Singapore is seeking a Data Engineer to test and validate data pipelines, ensuring data integrity and optimal performance. The ideal candidate will have strong Python skills, familiarity with AWS services, and a knack for debugging and data quality. This role offers opportunities to work with cloud-based infrastructure and collaborate with diverse teams.

Qualifications

• Proficient in Python with unit testing skills.
• Experience in data pipeline development and troubleshooting.
• Strong debugging and problem-solving skills.

Responsibilities

• Test and validate data pipelines for all new releases.
• Ensure data integrity and completeness after deployments.
• Debug and troubleshoot pipeline issues.
• Optimize pipeline performance and efficiency.

Skills

Python

AWS services (S3, Lambda, EC2, CloudWatch)

Debugging

Data pipeline development

Tools

Git

Job description

Responsibilities

• Test and validate data pipelines for all new releases.
• Ensure data integrity and completeness after deployments.
• Debug and troubleshoot pipeline issues.
• Implement fixes and document root causes.
• Optimise pipeline performance and efficiency.
• Write clean, maintainable Python code with unit tests.
• Collaborate with data scientists, engineers, and stakeholders.
• Support AWS-based pipeline infrastructure (e.g., S3, Lambda, EC2, CloudWatch).

Requirements

• Proficient in Python (unit testing, clean code practices).
• Experience in data pipeline development and troubleshooting.
• Familiarity with AWS services (S3, Lambda, EC2, CloudWatch).
• Strong debugging and problem-solving skills.
• Attention to detail and focus on data quality.

Good to have

• Basic understanding of data science workflows.
• Experience with CI/CD and version control (e.g., Git).