Data Engineer

StackTech Pte. Ltd.; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A dynamic tech company in Singapore is seeking a Data Engineer with over 2 years of experience to design and maintain scalable data pipelines. The role emphasizes building reliable data platforms on Google Cloud, focusing on both real-time and batch processing. Candidates should have strong skills in Python and SQL, along with familiarity in tools like BigQuery and Docker. The position offers opportunities for personal and professional growth in a cloud-native environment.

Benefits

Clear growth path towards Senior Data Engineer

Involvement in modern data platforms

Opportunities for professional development

Qualifications

• 2+ years of experience in data engineering or backend roles.
• Strong Python skills, including pandas and PySpark.
• Experience with building and maintaining production data pipelines.

Responsibilities

• Design and maintain batch and real-time data pipelines.
• Implement event-driven architectures for data processing.
• Monitor streaming pipelines for performance and reliability.

Skills

Python

SQL

Docker

Data modeling

Event-driven architectures

Education

Bachelor’s degree in Computer Science, Data Engineering, or related field

Tools

Google Cloud Platform

BigQuery

Pub/Sub

Cloud Run

Job description

Data Engineer

Location: Singapore

Team: Data & Analytics

Reports to: Head of Data / Data Engineering Lead

Role Overview

We are looking for a Data Engineer with 2+ years of relevant experience to design, build, and operate scalable, reliable, production-grade data platforms that support analytics, machine learning, and business decision-making.

This role covers both real-time data streaming and batch processing, with a strong focus on

engineering quality, system reliability, and data freshness, primarily on Google Cloud Platform (GCP).

Key Responsibilities

1. Data Pipelines & Platform Engineering (Batch & Streaming)

• Design, build, and maintain batch and real-time data pipelines
• Work with Pub/Sub, Cloud Run, and BigQuery
• Develop data processing logic using Python (pandas, PySpark) and SQL
• Build real-time ingestion services supporting:Low-latency ingestionIdempotency and de-duplicationData validation and schema evolution
• Implement layered data architectures:Raw → Curated → Analytics-ready datasets
• Handle late-arriving data, replays, and historical backfills

2. Real-Time Data Streaming & Processing

• Participate in designing event-driven architectures
• Implement streaming logic for:Real-time / near-real-time aggregationsOperational and monitoring datasets
• Understand and apply exactly-once or effectively-once processing semantics
• Monitor streaming pipelines for latency, throughput, and failures

3. Data Modeling & Data Warehousing

• Design and maintain analytics-optimized BigQuery data models
• Apply appropriate:PartitioningClustering
• Support high-ingestion-rate tables and high-performance analytical queries
• Ensure schema consistency across development and production environments

4. Analytics & Machine Learning Enablement

• Build high-quality datasets for:Reporting and dashboardsTime-series analysisMachine learning feature generation
• Collaborate with analysts and data scientists to:Understand data requirementsValidate data accuracy and freshness

5. Cloud Infrastructure & Engineering Practices

• Containerize data services using Docker
• Build and deploy via Cloud Build and Artifact Registry
• Operate Cloud Run services and scheduled jobs
• Assist with:Service accounts and IAM rolesSecrets and environment configuration
• Contribute to CI/CD automation and deployment workflows

6. Data Quality, Governance & Reliability

• Implement data quality checks for both streaming and batch pipelines
• Help identify and resolve:Data delaysMissing or duplicate dataSchema breaking changes
• Maintain documentation, including:Data dictionariesStreaming architecture diagramsOperational runbooks
• Ensure pipelines are auditable, reproducible, and reliable

Required Qualifications

Minimum Requirements

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field
• 2+ years of experience in data engineering, backend engineering, or data platform roles
• Strong Python skills (pandas and/or PySpark)
• Solid SQL skills (BigQuery experience preferred)
• Hands-on experience building or maintaining production data pipelines
• Understanding of batch vs real-time streaming data processing concepts

Technical Competencies

• Familiarity with event-driven architectures
• Understanding of data modeling and data warehouse design
• Experience handling schema evolution and historical backfills
• Basic performance, scalability, and cost-optimization awareness

Engineering & DevOps Skills

• Experience with Docker and containerized applications
• Familiarity with Git-based development workflows
• Exposure to CI/CD pipelines
• Ability to troubleshoot and debug production issues

Nice to Have

• Experience with real-time streaming systems (Pub/Sub, Kafka, Dataflow)
• Exposure to time-series or near-real-time analytics
• Familiarity with:Dataflow / Apache BeamVertex AIBI tools such as Tableau or Looker
• Experience working with multi-region or multi-currency datasets

What Success Looks Like

• Data pipelines run reliably and with low latency
• Streaming and batch datasets are consistent and trustworthy
• Data freshness SLAs are met
• Downstream analytics and ML teams confidently rely on the data platform

Why Join Us

• Work on modern real-time data platforms
• Clear growth path toward Senior Data Engineer
• Strong engineering ownership and technical depth
• Cloud-native environment focused on long‑term maintainability

数据工程师（Data Engineer）

工作地点：新加坡

团队：数据与分析团队

汇报对象：数据负责人 / 数据

岗位概述

我们正在招聘一名 具有 2 年及以上相关经验的数据工程师，负责设计、构建和维护 可扩展、稳定、生产级的数据平台，为数据分析、机器学习和业务决策提供可靠的数据支持。该岗位将同时覆盖 实时数据流（Real-time Streaming）与离线批处理（Batch Processing)，技术栈以 Google Cloud Platform（GCP) 为核心，强调 工程质量、系统稳定性与数据时效性。

工作职责

1. 数据管道与平台建设（批处理 + 实时流）

• 设计并维护 实时与离线数据管道
• 使用 Pub/Sub、Cloud Run、BigQuery
• 使用 Python（pandas、PySpark）与 SQL 进行数据处理
• 构建 实时数据接入服务，支持：低延迟写入幂等处理与去重数据校验与 Schema 演进
• 落地分层数据架构：原始层（Raw）→ 清洗层（Curated）→ 分析层（Analytics-ready）
• 处理 延迟数据、乱序数据、数据回放与历史补数

2. 实时数据流与流式处理

• 参与设计并实现 事件驱动架构
• 实现流式处理逻辑，包括：实时/准实时指标计算实时监控与运营数据集
• 理解并实践 Exactly-once 或 Effectively-once 处理语义
• 监控实时数据链路的延迟、吞吐量与异常情况

3. 数据建模与数据仓库

• 设计和维护 BigQuery 分析型数据模型
• 合理使用分区（Partitioning）与聚簇（Clustering）
• 支持高频写入与高性能分析查询
• 保证 开发环境与生产环境 Schema 一致性

4. 数据分析与机器学习支持

• 构建可复用的数据集，用于：报表与分析时间序列分析机器学习特征工程
• 与数据分析师、数据科学家协作：理解数据需求校验数据准确性与时效性

5. 云基础设施与工程化

• 使用 Docker 构建和部署数据服务
• 通过 Cloud Build / Artifact Registry 进行版本管理
• 运行 Cloud Run 服务与定时任务
• 配合完成：Service Account 与 IAM 权限配置环境变量与密钥管理
• 参与 CI/CD 自动化流程建设

6. 数据质量、治理与稳定性

• 实施数据质量校验与监控
• 协助发现并处理：数据延迟数据缺失Schema 变更风险
• 维护数据文档、数据流图与运维说明
• 确保数据 可追溯、可复现

任职要求

基本要求（必须）

• 计算机科学、数据工程、信息系统或相关技术专业本科及以上学历
• 2 年及以上数据工程 / 后端 / 数据平台相关工作经验
• 熟练使用 Python（pandas / PySpark 至少其一）
• 熟练编写 SQL（有 BigQuery 经验优先）
• 有 真实生产环境 的数据管道建设或维护经验
• 理解 批处理与实时流处理 的基本原理
• 熟悉事件驱动架构与流式处理思路
• 理解数据建模与数据仓库设计
• 能处理 Schema 演进与历史数据补数
• 具备基础的性能与成本意识

工程能力要求

• 具备 Docker 使用经验
• 熟悉 Git 基本工作流
• 能配合 CI/CD 流程完成部署
• 具备基础的问题定位与排查能力

加分项（Nice to Have）

• 有 实时数据流系统 经验（Pub/Sub / Kafka / Dataflow）
• 有时间序列或准实时分析经验
• 熟悉：Dataflow / Apache BeamVertex AIBI 工具（Tableau / Looker）
• 有多区域或多币种数据处理经验

成功标准

• 能稳定交付 可运行、可维护的数据管道
• 实时与离线数据链路稳定、可监控
• 数据质量满足分析与下游使用要求
• 能独立承担中等复杂度的数据工程任务

为什么加入我们

• 深入参与 实时数据平台与核心数据系统建设
• 技术成长路径清晰，工程实践扎实
• 有机会向高级数据工程师 / 技术专家发展
• 云原生技术栈，强调工程质量与长期维护