Data Engineer

FPT Asia Pacific; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading technology firm in Singapore is seeking a skilled Data Engineer to support data engineering tasks and manage data platforms. The ideal candidate will have a degree in Computer Science or related fields and experience in data pipeline implementation, data modeling, and cloud technologies. Proficiency in Python and database systems is essential, as well as effective communication skills. This role offers opportunities for professional development in a collaborative environment.

Qualifications

• Experienced in data engineering, including implementation of data pipelines and development of data models.
• Familiar with architecting, designing, and developing data platforms.
• Experience with various programming languages and tools for data manipulation and analysis.

Responsibilities

• Support data engineering tasks, including enhancement of data pipelines.
• Manage the data platform and perform regular software version upgrades.
• Handle application configuration changes and manage user access requests.

Skills

Data engineering

Python

Database management

ETL processes

Team collaboration

Effective communication

Education

Degree in Computer Science, Engineering, or related disciplines

Tools

Python packages (e.g. Pandas, GeoPandas, Shapely)

CI/CD tools (e.g. Jenkins, GitLab)

Infrastructure-as-code Tools (e.g. Terraform, CloudFormation)

Log Management tools (e.g. Azure Monitor, AWS CloudWatch)

Job description

• Support data engineering tasks, including the implementation and enhancement of data pipelines, as well as the rectification of broken pipelines.
• Manage the data platform and perform regular software version upgrades across all environments, ensuring thorough testing and detailed documentation.
• Support daily operational needs by handling application configuration changes, managing user access request to application, addressing general function queries, and troubleshooting on issues.
• Perform source code review and configuration review periodically to ensure code quality and verify that sensitive information, such as secrets, is not hardcoded or embedded in source codes or configuration files.
• Periodic patching of Azure Cloud Servers
• Develop business processes by engaging stakeholders to understand use cases for building data pipelines, performing data modeling, completing data collection forms, documenting use cases, defining data attributes within various data quality zones, and establishing naming conventions for datasets.

Requirements:

• Degree in Computer Science, Engineering, or related disciplines
• Experienced in data engineering, including the implementation of data pipelines, development of data models and schemas, and pipeline monitoring and management.
• Experienced in architecting, designing, and developing data platform.

Experience with:

• Python, PySpark, or similar programming languages
• Python packages for data manipulation and analysis (e.g. Pandas, GeoPandas, Shapely)
• Database design and management (e.g. PostgreSQL, MS SQL, Oracle, Geodatabase)
• Scripting and version control (e.g. Bash, PowerShell, Git)
• ETL (Extract, Transform, Load) processes
• Technologies such as JupyterHub, RStudio, PowerBI
• CI/CD tools such as Jenkins, GitLab, YAML
• API development and SFTP for secure data transfer
• Able to work well with a team and be willing to learn.
• Effective presentation, communication and writing skills.

Added advantage with the following:

• Infrastructure-as-code Tools Terraform, CloudFormation
• Log Management tools (e.g. Azure Monitor, AWS CloudWatch, Splunk)