Senior Data Engineer (AVP) – Real-Time Data & AI Pipelines

OCBC Group; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

OCBC Group in Singapore is seeking a Data Engineer (AVP) to build, maintain, and improve data pipelines powering an enterprise data warehouse and AI knowledge base within a banking environment.

You’ll work with senior engineers and the head of data engineering to transform structured and unstructured data into reliable datasets and services that support risk management, customer engagement, fraud detection, and intelligent automation.

Qualifications

• Bachelor’s/Master’s in CS or related field.
• 5+ years in data engineering or related roles.
• Experience with data warehouses, lakehouses, and batch/real-time processing.
• Familiarity with AI knowledge bases, vector databases, and semantic search is a plus.
• Strong product mindset and ability to treat data as a product.

Responsibilities

• Batch & streaming data pipeline development.
• Build and maintain pipelines feeding data warehouses/lakehouses (Cloudera, BigQuery, Redshift).
• Develop data models for reusable analytical datasets and reporting foundations.
• Follow SLAs and monitoring, troubleshoot pipeline issues.
• Develop batch jobs using Spark, SQL, Python or Java.
• Support real-time streaming pipelines using Flink or similar tools.
• Build and operate CDC pipelines (Debezium, Confluent, Fivetran).
• Create ingestion pipelines from APIs, GA4, and other sources.
• Implement messaging/streaming via Pub/Sub and Kafka.
• Write clean SQL and Python for data ingestion/processing.
• Develop Airflow DAGs for workflows; follow orchestration best practices.
• Deploy data workloads on Cloud (GCP/AWS) with Docker/Kubernetes/Cloud Run.
• Contribute to CI/CD pipelines; use Terraform for infra provisioning.
• Build REST APIs and backend services (Python/Flask); use Redis as needed.
• Assist with vector DB pipelines and embedding generation for AI/ RAG.
• Support semantic search and retrieval quality for AI-facing data layers.
• Collaborate with Risk/Marketing/Operations; translate business requirements into tasks.

Skills

Communication

Team collaboration

Problem solving

Adaptability

Education

Bachelor’s degree in Computer Science or related field

Tools

Cloudera

BigQuery

Redshift

Spark

SQL

Python

Airflow

Terraform

Docker/Kubernetes/Cloud Run

Job description

OCBC Group in Singapore is seeking a Data Engineer (AVP) to build, maintain, and improve data pipelines powering an enterprise data warehouse and AI knowledge base within a banking environment.

You’ll work with senior engineers and the head of data engineering to transform structured and unstructured data into reliable datasets and services that support risk management, customer engagement, fraud detection, and intelligent automation.