Senior Data Engineer: Spark Streaming & ML Pipelines

Visa Hunt; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Visa Hunt is seeking a senior data engineer to design, build, and maintain high‑performance data pipelines powering analytics and machine learning products. You will collaborate with data scientists, product, and infrastructure teams to turn raw data into scalable, reliable assets.

In this role you will architect end-to-end batch and Spark Streaming pipelines on cloud platforms, implement ML feature pipelines and realtime inference, and enforce data quality, governance, and security at petabyte

Qualifications

• 5+ years production-grade data engineering experience.
• Deep expertise with Apache Spark (batch & streaming) and distributed processing.
• Hands-on ML pipeline experience (feature engineering, model training, deployment).
• Experience with cloud data platforms and data warehousing.
• Strong SQL and Python/Scala; familiarity with Airflow, dbt, or similar.

Responsibilities

• Architect end-to-end batch and Spark Streaming pipelines on cloud (AWS/GCP/Azure).
• Implement ML feature pipelines and realtime inference services.
• Optimize petabyte-scale processing with Spark, Kafka, and Flink.
• Build and maintain data warehouses/lakes (Redshift, BigQuery, Snowflake).
• Enforce data quality, governance, and security.
• Develop CI/CD, monitoring, and alerting for pipelines.
• Mentor engineers and drive best-practice documentation.

Skills

Apache Spark

Kafka

Distributed processing

SQL

Python/Scala

Airflow

Tools

Redshift

BigQuery

Snowflake

Job description

Visa Hunt is seeking a senior data engineer to design, build, and maintain high‑performance data pipelines powering analytics and machine learning products. You will collaborate with data scientists, product, and infrastructure teams to turn raw data into scalable, reliable assets.

In this role you will architect end-to-end batch and Spark Streaming pipelines on cloud platforms, implement ML feature pipelines and realtime inference, and enforce data quality, governance, and security at petabyte