Data Engineer

HORIZON SOFTWARE PTE. LTD.; Singapore; On-site; SGD 150,000 - 210,000; Full time

Job summary

Horizon Software Pte. Ltd. in Singapore is seeking an experienced Data Engineer to design, build and optimize large-scale data pipelines.

You will collaborate with data scientists and analysts to deliver reliable data products and ensure data quality. The role requires 8+ years in data engineering, strong SQL and Python, cloud platforms, Spark/ETL, and data warehousing experience to implement governance and security practices.

Qualifications

• 8+ years of data engineering experience.
• Strong SQL and Python programming skills.
• Experience with Spark and ETL frameworks.
• Hands-on experience with cloud data platforms.
• Knowledge of data warehousing concepts.
• Experience with workflow orchestration tools.
• Understanding of data modeling and optimization.
• Familiarity with Agile methodologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines.
• Build and optimize ETL/ELT workflows.
• Develop and manage data warehouses and data lakes.
• Integrate data from multiple sources.
• Optimize database performance and data quality.
• Collaborate with data scientists, analysts, and application teams.
• Implement data governance and security best practices.
• Monitor and troubleshoot data pipeline issues.
• Automate data processing workflows.
• Maintain technical documentation.

Skills

Data Engineering

ETL/ELT Development

Data Warehousing

Cloud Platforms

Big Data Technologies

Database Management

Performance Optimization

Data Modeling

Tools

Python

SQL

Scala

Apache Spark

Hadoop

Kafka

Snowflake

Redshift

BigQuery

AWS

Azure

Google Cloud Platform

Airflow

Databricks

SQL Server

PostgreSQL

MySQL

Git

Docker

Kubernetes

Job description

Job Description & Requirements

Key Responsibilities

• Design, develop, and maintain scalable data pipelines.
• Build and optimize ETL/ELT workflows.
• Develop and manage data warehouses and data lakes.
• Integrate data from multiple sources.
• Optimize database performance and data quality.
• Collaborate with data scientists, analysts, and application teams.
• Implement data governance and security best practices.
• Monitor and troubleshoot data pipeline issues.
• Automate data processing workflows.
• Maintain technical documentation.

Technical Skills

• Python, SQL, Scala
• Apache Spark, Hadoop, Kafka
• ETL/ELT Development
• Data Warehousing
• Snowflake, Redshift, BigQuery
• AWS, Azure, Google Cloud Platform
• Airflow, Databricks
• SQL Server, PostgreSQL, MySQL
• Git, Docker, Kubernetes

Mandatory Requirements

• 8+ years of Data Engineering experience.
• Strong SQL and Python programming skills.
• Experience with Spark and ETL frameworks.
• Hands-on experience with cloud data platforms.
• Knowledge of data warehousing concepts.
• Experience with workflow orchestration tools.Understanding of data modeling and optimization.
• Familiarity with Agile methodologies.

Core Competencies

• Data Engineering
• ETL/ELT Development
• Data Warehousing
• Cloud Platforms
• Big Data Technologies
• Database Management
• Performance Optimization
• Data Modeling