Data Engineer

FPT Asia Pacific; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading technology company in Singapore seeks a Data Engineer to manage and enhance data pipelines, conduct software upgrades, and engage with stakeholders for data modeling. The ideal candidate has a degree in Computer Science or related fields and a keen interest in pure Data Engineering. Proficiency in Python and SQL, as well as experience with Azure and ETL processes, is required. This role emphasizes collaboration and proactive issue resolution.

Qualifications

• Degree in Computer Science, Engineering, or related disciplines.
• Strong interest in pure Data Engineering work.
• Experienced in implementing and enhancing data pipelines.

Responsibilities

• Support data engineering tasks and manage data pipelines.
• Conduct source code and configuration reviews.
• Engage stakeholders to understand use cases.

Skills

Python

SQL

ETL

Data modeling

Cloud Platforms

Docker

Education

Degree in Computer Science or Engineering

Tools

Azure Data Factory

PostgreSQL

Docker

PowerBI

Job description

• Support data engineering tasks, including the implementation, enhancement, and setup of data pipelines, as well as the rectification of broken pipelines.
• Manage the data platform and perform regular software version upgrades across environments, ensuring thorough testing and clear documentation.
• Support daily operational needs by handling application configuration changes, managing user access, addressing functional queries, and troubleshooting issues.
• Perform source code reviews and configuration reviews periodically to ensure code quality and to verify that sensitive information is not hardcoded or embedded in source code or configuration files.
• Conduct periodic patching of Azure Cloud Servers.
• Engage stakeholders to understand use cases for building new data pipelines, performing data modeling, completing data collection forms, documenting use cases, defining data attributes within various data quality zones, and establishing dataset naming conventions.
• Demonstrate a proactive and diligent approach to issue resolution, improvements, and ownership of data engineering tasks.
• Collaborate effectively within the team and communicate clearly with stakeholders.

Technical Skills

• Programming & Data Manipulation - Python, PySpark, or similar programming languages. Python packages such as Pandas, GeoPandas, Shapely.
• Databases & SQL - Experience with database design and management, e.g. PostgreSQL, MS SQL, Oracle, Geodatabase. Strong SQL programming skills (complex queries, performance tuning, data manipulation).
• ETL & Scripting - ETL (Extract, Transform, Load) experience. Scripting and version control (Bash, PowerShell, Git).
• Analytics & Tools - Technologies such as JupyterHub, RStudio, PowerBI. CI/CD tools such as Jenkins, GitLab, YAML.
• Integration & Cloud - API development and SFTP for secure data transfer. Cloud Platforms: Microsoft Azure, AWS. Cloud Technologies: Azure Data Factory, Databricks, Azure Functions, Azure Key Vault, AWS Lambda.
• Containerization - Experience with Docker, Kubernetes (added advantage).

Requirements

• Degree in Computer Science, Engineering, or related disciplines.
• Strong interest in pure Data Engineering work (not AI research or Business Analytics roles).
• Experienced in data engineering, including:
• Implementation and enhancement of data pipelines
• Development of data models and schemas
• Pipeline monitoring and management
• Setting up data pipelines on Azure, Databricks, and using Python scripting
• Experienced in architecting, designing, and developing data platforms.
• Good communication skills and ability to quickly understand technical and business requirements.
• Proactive, diligent, and able to work independently as well as collaboratively.