Data Engineer

OPUS IT Services Pte Ltd; Hong Kong; On-site; HKD 1,110,000 - 1,481,000; Full time

Job summary

OPUS IT Services Pte Ltd in Singapore seeks a Data Engineer to design, build, and operate scalable data platforms and ETL pipelines for analytics, reporting, and AI-driven decision-making across hybrid and cloud environments.

You will collaborate with data architects and stakeholders to deliver end-to-end data solutions, ensure security compliance per government standards, and support MINDEF and GCC cloud initiatives.

Qualifications

• 3+ years of experience in enterprise data engineering
• Candidate must have CAT 1 / G50 security clearance
• Prior MINDEF project experience (mandatory)

Responsibilities

• Design and build scalable data pipelines
• Develop enterprise data platforms including warehouses/lakes/lakehouses and GenAI-ready data platforms
• Implement cloud, hybrid, and on-premises data architectures with enterprise standards
• Design, develop, and maintain ETL/ELT pipelines using PySpark, Spark SQL, and Talend
• Ensure data quality, validation, and error-handling frameworks across pipelines
• Collaborate with data architects, stakeholders, and infrastructure teams to deliver end-to-end data solutions across cloud and hybrid environments

Skills

Data pipelines

PySpark

SQL

Python

Databricks

CI/CD testing

Data warehousing

Security clearance

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or related field

Tools

Talend

Oracle/SQL Server

IBM MQ

Job description

• Employment type -1-year contract (Renewable)
• Eligibility:Singaporean ONLY (require security clearance)
• Location: Ang Mo Kio
• Working Days/ Hours: Mon-Fri: 8.30am to 6pm

Role Overview

As a Data Engineer, you will design, build, and operate scalable, secure, and high-performance enterprise data platforms and ETL pipelines that enable analytics, reporting, and data-driven as well as AI-driven decision-making.

You will play a key technical role in delivering end-to-end data pipelines, analytics platforms, integration solutions, and generative AI solutions across hybrid and cloud environments, including Government Commercial Cloud (GCC) and MINDEF Commercial Cloud (MCC).

You will work closely with data and solution architects, business stakeholders, and infrastructure teams to translate requirements into robust, production-grade data solutions.

Job Responsibilities

1. Data Engineering & Platform Design

• Design and develop scalable and reliable data pipelines.
• Build enterprise data platforms including data warehouses, data lakes, lakehouses, and AI-ready data platforms that support GenAI workloads (e.g., real-time ingestion, vector search, semantic retrieval).
• Implement cloud, hybrid, and on-premises data architectures aligned with enterprise standards.

2. Data Integration & ETL/ELT

• Design, develop, optimize, and maintain scalable ETL/ELT data pipelines using PySpark, Spark SQL, and modern data processing frameworks. Experience with Talend is preferred.
• Build and support data pipelines that enable Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and AI analytics use cases.
• Integrate structured, semi-structured, and unstructured data sources.
• Implement and maintain data quality, validation, and error-handling frameworks.

3. Application Migration Testing, Validation & Test Automation

• Analyze migrated applications, application flows, interfaces, APIs, databases, and integration dependencies to determine testing scope and coverage.
• Develop key test assets, including functional, integration, regression, and production verification test cases, test data requirements, and Requirements Traceability Matrix (RTM).
• Execute and automate testing where feasible, including smoke, functional, regression, integration, API, and batch validation testing.
• Support application modernization initiatives by validating business functionality, interfaces, and system integrations post-migration.
• Leverage AI-assisted tools to accelerate application discovery, test scenario generation, test automation, and coverage analysis.
• Perform security fix verification and regression testing to validate remediation changes, ensure business continuity, and support defect management and closure.
• Produce testing deliverables including Test Strategy, Test Suites, Test Execution Reports, Defect Reports, Coverage Reports, and UAT Readiness Reports.

4. Storage & Data Management

• Design and manage data storage solutions with a strong emphasis on modern data platforms, data quality, and data governance.
• Implement and maintain data lifecycle management and cost-optimization strategies.
• Ensure data lineage, traceability, and auditability.

5. Analytics Enablement & BI Integration

• Engineer analytics-ready data models for reporting and self-service analytics.
• Enable BI Dashboards, Reports, and analytics through performant and governed data pipelines.

6. Security, Governance & Compliance

• Ensure compliance with government security and privacy requirements.
• Implement and maintain encryption, access controls, and secure data handling practices.
• Implement and maintain security policies and procedures for the system.

Job Requirements

Education

• Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.

Core Technical Skills

• Strong hands-on data engineering experience.
• Functional, regression, integration, and production validation testing.
• API, interface, database, MQ (IBM MQ preferred), and batch dependency analysis.
• Relational databases (Oracle, SQL Server, PostgreSQL, DB2).
• CI/CD-integrated testing frameworks.
• Expertise in ETL/ELT with Talend; hands-on experience preferred.
• Experience with data warehouses, data lakehouses, data lakes, and object storage systems. Hands-on experience with Databricks preferred.
• Strong SQL, Java, and Python (PySpark) skills.

Experience & Security Requirements

• Minimum 3+ years of experience in enterprise data engineering roles
• Prior CAT 1 / G50 security clearance (mandatory)
• Previous experience delivering MINDEF projects (mandatory)