Data Engineer (Smart TV OS)

JonDavidson Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

JonDavidson Pte Ltd seeks an experienced Senior Data Engineer to lead end-to-end real-time data ingestion and streaming data warehouse design. You will own ingestion pipelines for high-scale logs, user behavior data, and operational logs, ensuring low latency and data quality.

You will work across cloud analytics platforms (GCP BigQuery, Azure, Databricks) and streaming frameworks (Kafka, Spark, Flink) to support dashboards and analytics engines with cross-border collaboration.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• 3+ years of professional big data engineering experience in high-scale environments.
• Experience building real-time data pipelines and streaming data warehouses.

Responsibilities

• Own end-to-end ingestion pipelines for real-time data.
• Design and tune multi-layer real-time data models and streaming ETL.
• Maintain Kafka infrastructure and resolve latency, backlogs, and deduplication issues.
• Develop and optimize Spark and Flink tasks for sub-second latency.
• Collaborate with data product owners, analysts, and stakeholders to deliver scalable pipelines.
• Establish data quality standards, monitoring, alerts, and SLA governance.

Education

Bachelor’s degree in Computer Science or related field

Tools

Kafka

Spark

Flink

CDC

Binlog

GCP BigQuery

Azure

Databricks

dbt

Job description

We are seeking an experienced Senior Data Engineer to join our fast-growing global team. In this role, you will lead the end-to-end design, construction, and iteration of our real-time business data ingestion systems and streaming data warehouse architecture.

You will work at the intersection of high-scale real-time data streaming, cloud analytics platforms, and business-critical analytics engines to support real-time dashboards, user profiling platforms, and business monitoring systems.

Key Responsibilities

1. Ingestion Pipeline Ownership: Own the lifecycle development of real-time collection, cleaning, and warehousing of business logs, user behavior data, and database operational logs to guarantee pipeline stability, completeness, and low latency.
2. Real-Time Data Warehouse Architecture: Design multi-layer real-time data models, build and tune streaming ETL pipelines, and support continuous delivery of real-time business metrics and analytics.
3. Kafka & Streaming Optimization: Maintain and optimize core Kafka streaming infrastructure. Diagnose and resolve online challenges including message backlogs, data skew, message loss, duplication, and partition tuning.
4. Spark & Flink Operations: Develop, tune, and operate real-time Spark/Flink tasks to maximize computing resource utilization, throughput, and sub-second latency.
5. Cross-Functional Collaboration: Partner with data product managers, analysts, and business stakeholders to translate complex business requirements into scalable, production-ready data pipelines.
6. Data Quality & Governance: Establish data standardization practices, automated data quality monitoring, alert thresholds, and SLA governance protocols.

Required Experience & Qualifications

• Education: Bachelor’s degree or higher in Computer Science, Software Engineering, Data Science, or a related technical discipline.
• Work Experience: 3+ years of professional big data engineering experience in high-scale tech/internet environments with proven track records in real-time pipeline construction.
• Core Technical Stack:
• Languages: Proficiency in Python, Scala, or Java with solid Shell/Linux scripting capabilities.
• Streaming Infrastructure: Expert-level knowledge of Apache Kafka (partitioning strategies, consumer mechanisms, lag mitigation, deduplication).
• Compute & Processing: Strong hands-on experience with Apache Spark (Spark SQL) and Apache Flink (Flink SQL) for streaming task optimization.
• Ingestion & CDC: Hands-on experience with Change Data Capture (CDC) and Binlog real-time synchronization from operational databases.
• Cloud Ecosystems: Exposure to cloud-native platforms (Google Cloud Platform / BigQuery, Azure, Databricks, dbt).
• Languages & Communication: Professional working fluency in English (verbal and written) for cross-border collaboration.

Preferred Qualifications

• Prior experience studying or working internationally or collaborating across cross-border remote tech teams.
• Hands-on knowledge of real-time SLA governance, incident post-mortem processes, and automated alerting frameworks.