M06 - Data Engineer

FPT ASIA PACIFIC PTE. LTD.; Singapore; On-site; SGD 60,000 - 80,000; Full time

Job summary

A technology company in Singapore is seeking a Data Engineer to implement and enhance data pipelines, manage data platforms, and engage with stakeholders for data modeling. The ideal candidate has a degree in Computer Science or Engineering, strong programming skills in Python and SQL, and a proactive approach to problem-solving. Experience with cloud technologies like Azure is essential. This role offers the opportunity to work on critical data engineering tasks in a collaborative environment.

Qualifications

• Strong interest in pure Data Engineering work and architecture of data platforms.
• Experience monitoring and managing data pipelines.
• Good communication skills to understand requirements.

Responsibilities

• Support data engineering tasks including data pipeline setup.
• Manage and upgrade data platforms.
• Document use cases and ensure code quality.

Skills

Python

PySpark

SQL

ETL

Azure Data Factory

Docker

Kubernetes

Education

Degree in Computer Science, Engineering, or related disciplines

Tools

PostgreSQL

MS SQL

GeoPandas

JupyterHub

PowerBI

Job description

Data Engineering Work (Key Responsibilities)

• Support data engineering tasks, including the implementation, enhancement, and setup of data pipelines, as well as the rectification of broken pipelines.
• Manage the data platform and perform regular software version upgrades across environments, ensuring thorough testing and clear documentation.
• Support daily operational needs by handling application configuration changes, managing user access, addressing functional queries, and troubleshooting issues.
• Perform source code reviews and configuration reviews periodically to ensure code quality and to verify that sensitive information is not hardcoded or embedded in source code or configuration files.
• Conduct periodic patching of Azure Cloud Servers.
• Engage stakeholders to understand use cases for building new data pipelines, performing data modeling, completing data collection forms, documenting use cases, defining data attributes within various data quality zones, and establishing dataset naming conventions.
• Demonstrate a proactive and diligent approach to issue resolution, improvements, and ownership of data engineering tasks.
• Collaborate effectively within the team and communicate clearly with stakeholders.

Technical Skills

• Programming & Data Manipulation - Python, PySpark, or similar programming languages. Python packages such as Pandas, GeoPandas, Shapely.
• Databases & SQL - Experience with database design and management, e.g. PostgreSQL, MS SQL, Oracle, Geodatabase. Strong SQL programming skills (complex queries, performance tuning, data manipulation).
• ETL & Scripting - ETL (Extract, Transform, Load) experience. Scripting and version control (Bash, PowerShell, Git).
• Analytics & Tools - Technologies such as JupyterHub, RStudio, PowerBI. CI/CD tools such as Jenkins, GitLab, YAML.
• Integration & Cloud - API development and SFTP for secure data transfer. Cloud Platforms: Microsoft Azure, AWS. Cloud Technologies: Azure Data Factory, Databricks, Azure Functions, Azure Key Vault, AWS Lambda.
• Containerization - Experience with Docker, Kubernetes (added advantage).

Requirements

• Degree in Computer Science, Engineering, or related disciplines.
• Strong interest in pure Data Engineering work (not AI research or Business Analytics roles).
• Experienced in data engineering, including:
  • Implementation and enhancement of data pipelines
  • Development of data models and schemas
  • Pipeline monitoring and management
  • Setting up data pipelines on Azure, Databricks, and using Python scripting
• Experienced in architecting, designing, and developing data platforms.
• Good communication skills and ability to quickly understand technical and business requirements.
• Proactive, diligent, and able to work independently as well as collaboratively.