Senior Data Engineer: Build Scalable AI/GenAI Pipelines

Morgan McKinley; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Morgan McKinley in Singapore is seeking a Senior Data Engineer to architect, build, and optimize high‑throughput data pipelines powering analytics, AI, and GenAI use cases.

You will collaborate with data scientists, analysts, and system owners to translate complex requirements into secure, automated, production‑grade data platforms in a hybrid cloud environment. The role emphasizes reliability, security, release coordination, and strong ownership of production pipelines.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering or cloud-scale analytics with ownership of production pipelines.
• Strong ability to design, build, optimise, and operate production-grade batch/streaming pipelines.
• Hands-on expertise in Python and SQL for data transformation and validation.
• Familiarity with Apache Spark (PySpark) and large-scale data processing.
• Experience implementing knowledge bases and RAG solutions for AI use cases.
• Strong problem-solving, independence, and ability to lead technical discussions.

Responsibilities

• Build scalable ingestion, transformation, and serving capabilities across a hybrid cloud environment, ensuring reliability, security, and compliance with enterprise standards.
• Optimise batch and streaming workflows using Databricks and Kafka, applying best practices for performance, resilience, and maintainability.
• Deliver high-quality, secure data to support reporting, machine learning, and GenAI use cases.
• Lead the setup and operationalisation of knowledge bases and retrieval-augmented generation stacks for agentic AI applications.
• Perform cleansing and transformation using PySpark or SQL based on business requirements.
• Monitor, troubleshoot, and ensure pipeline reliability, while owning production readiness, observability, incident triage, and release coordination.
• Provide guidance on reusable components, code quality, deployment readiness, and production support practices.
• Connect diverse data sources (files, APIs, databases, streams), maintain metadata, and ensure transparency through documentation.
• Work with Microsoft Fabric, Databricks, Delta Lake, and other platform components.
• Drive CI/CD workflows, version control, and enforce data governance, security, and compliance policies.

Skills

Python

SQL

PySpark

Data pipelines

CI/CD

Data pipelines

Cloud security

CI/CD

Stakeholder communication

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Databricks

Delta Lake

Kafka

Microsoft Fabric

Job description

Morgan McKinley in Singapore is seeking a Senior Data Engineer to architect, build, and optimize high‑throughput data pipelines powering analytics, AI, and GenAI use cases.

You will collaborate with data scientists, analysts, and system owners to translate complex requirements into secure, automated, production‑grade data platforms in a hybrid cloud environment. The role emphasizes reliability, security, release coordination, and strong ownership of production pipelines.