Data Engineer

Horizon Computer Management; Singapore; On-site; SGD 120,000 - 160,000; Full time

Job summary

Horizon Computer Management in Singapore is seeking an experienced Data Engineer with 7+ years to design, build, and maintain scalable data pipelines and infrastructure. You will develop ETL/ELT processes, manage large datasets, and support analytics and BI initiatives.

You will work with cloud platforms, data warehouses, and streaming technologies to ensure data quality, governance, and reliable data delivery for data scientists and business stakeholders.

Qualifications

• Bachelor's degree in Computer Science, IT, Data Engineering, or related field.
• Minimum 7 years of relevant experience in Data Engineering.
• Proven experience developing and managing enterprise-scale data pipelines.
• Experience with cloud data platforms and big data technologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines and data integration solutions.
• Build and manage ETL/ELT workflows for structured and unstructured data.
• Develop data models and optimise data warehouse solutions.
• Integrate data from multiple sources, including databases, APIs, and cloud platforms.
• Implement data processing solutions using batch and real-time streaming technologies.
• Ensure data quality, accuracy, consistency, and reliability.
• Work with data scientists, analysts, architects, and business stakeholders.
• Develop and optimise SQL queries and database performance.
• Implement data governance, security, and access control practices.
• Monitor, troubleshoot, and resolve data pipeline failures and performance issues.
• Work with cloud-based data platforms and distributed processing frameworks.
• Maintain technical documentation and follow software development best practices.

Skills

SQL

Python

ETL/ELT

Data pipelines

Spark/PySpark

Airflow

Cloud platforms

Snowflake/Redshift/BigQuery

SQL tuning

Kafka

Education

Bachelor's degree in CS/IT/Data Engineering

Tools

Airflow

Git

CI/CD

Hadoop

Spark

PySpark

Job description

Job Summary

We are looking for an experienced Data Engineer with 7+ years of experience in designing, developing, and maintaining scalable data pipelines and data infrastructure. The candidate will be responsible for building ETL/ELT processes, managing large datasets, optimising data workflows, and supporting data analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data integration solutions.

• Build and manage ETL/ELT workflows for structured and unstructured data.

• Develop data models and optimise data warehouse solutions.

• Integrate data from multiple sources, including databases, APIs, and cloud platforms.

• Implement data processing solutions using batch and real-time streaming technologies.

• Ensure data quality, accuracy, consistency, and reliability.

• Work with data scientists, analysts, architects, and business stakeholders.

• Develop and optimise SQL queries and database performance.

• Implement data governance, security, and access control practices.

• Monitor, troubleshoot, and resolve data pipeline failures and performance issues.

• Work with cloud-based data platforms and distributed processing frameworks.

• Maintain technical documentation and follow software development best practices.

Technical Skills & Requirements

• Strong proficiency in SQL and Python.

• Hands-on experience with ETL/ELT tools and data pipeline development.

• Experience with Apache Spark, PySpark, and Hadoop.

• Knowledge of Apache Airflow or similar workflow orchestration tools.

• Experience with data warehouses such as Snowflake, Amazon Redshift, Google BigQuery, or Azure Synapse.

• Hands-on experience with AWS, Microsoft Azure, or Google Cloud Platform.

• Knowledge of relational and NoSQL databases.

• Experience with data integration, data modelling, and data transformation.

• Familiarity with Kafka or other real-time data streaming technologies.

• Experience with Git, CI/CD, and automated deployment processes.

• Understanding of data governance, data security, and data quality frameworks.

Education & Experience

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related discipline.

• Minimum 7 years of relevant experience in Data Engineering.

• Proven experience developing and managing enterprise-scale data pipelines.

• Experience working with cloud data platforms and big data technologies.

Required Skills

• Strong analytical and problem-solving abilities.

• Excellent SQL and programming skills.

• Ability to handle large and complex datasets.

• Strong communication and collaboration skills.

• Ability to work independently and manage multiple priorities.

Preferred Certifications

• AWS Certified Data Engineer.

• Microsoft Certified: Azure Data Engineer Associate.

• Google Cloud Professional Data Engineer.

• Databricks Data Engineer Certification.