Senior Data Engineer

Visa Hunt; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Visa Hunt is seeking a senior data engineer to design, build, and maintain high‑performance data pipelines powering analytics and machine learning products. You will collaborate with data scientists, product, and infrastructure teams to turn raw data into scalable, reliable assets.

In this role you will architect end-to-end batch and Spark Streaming pipelines on cloud platforms, implement ML feature pipelines and realtime inference, and enforce data quality, governance, and security at petabyte

Qualifications

• 5+ years production-grade data engineering experience.
• Deep expertise with Apache Spark (batch & streaming) and distributed processing.
• Hands-on ML pipeline experience (feature engineering, model training, deployment).
• Experience with cloud data platforms and data warehousing.
• Strong SQL and Python/Scala; familiarity with Airflow, dbt, or similar.

Responsibilities

• Architect end-to-end batch and Spark Streaming pipelines on cloud (AWS/GCP/Azure).
• Implement ML feature pipelines and realtime inference services.
• Optimize petabyte-scale processing with Spark, Kafka, and Flink.
• Build and maintain data warehouses/lakes (Redshift, BigQuery, Snowflake).
• Enforce data quality, governance, and security.
• Develop CI/CD, monitoring, and alerting for pipelines.
• Mentor engineers and drive best-practice documentation.

Skills

Apache Spark

Kafka

Distributed processing

SQL

Python/Scala

Airflow

Tools

Redshift

BigQuery

Snowflake

Job description

Design, build, and maintain high performance data pipelines that power analytics and machine learning products. Collaborate with data scientists, product, and infrastructure teams to turn raw data into scalable, reliable assets.

Key Responsibilities

• Architect end to end batch and Spark Streaming pipelines on cloud (AWS/GCP/Azure).
• Implement ML feature pipelines and Realtime inference services.
• Optimize peta byte scale processing with Spark, Kafka, and Flink.
• Build and maintain data warehouses/lakes (Redshift, BigQuery, Snowflake).
• Enforce data quality, governance, and security.
• Develop CI/CD, monitoring, and alerting for pipelines.
• Mentor engineers and drive best practice documentation.

Required Experience

• 5+ years production grade data engineering.
• Deep expertise with Apache Spark (batch & streaming), Kafka, and distributed processing.
• Handson ML pipeline experience (feature engineering, model training, deployment).
• Cloud data platforms and warehousing.
• Strong SQL + Python/Scala; familiar with Airflow, dbt, or similar.