Senior Data Engineer – Scalable Pipelines for AI & GenAI

MORGAN MCKINLEY PTE. LTD.; Singapore; On-site; SGD 110,000 - 160,000; Full time

Job summary

Morgan McKinley Pte Ltd is seeking a Senior Data Engineer to architect, develop, and refine high-throughput data systems powering analytics, AI, and RAG solutions.

You will build scalable ingestion/processing across hybrid cloud, optimise batch/streaming workflows with Databricks and Kafka, and deliver secure data for reporting, ML, and GenAI use cases. Strong Python/SQL and Spark experience required.

Qualifications

• Bachelor's degree in CS, engineering, or related field.
• 5+ years in data engineering or cloud-scale analytics with production pipelines.
• Experience with Python and SQL for data transformation and validation.
• Hands-on with Spark (PySpark) and large-scale processing.
• Experience with knowledge bases and RAG for AI use cases.
• Strong problem-solving and independent work ethic.

Responsibilities

• Build scalable ingestion, transformation, and serving across hybrid cloud environments.
• Optimise batch and streaming workflows with Databricks and Kafka.
• Deliver secure data for reporting, ML, and GenAI use cases.
• Lead setup of knowledge bases and RAG stacks for AI apps.
• Ensure pipeline reliability, observability, and production readiness.
• Drive CI/CD, version control, and data governance.

Skills

Python

SQL

Apache Spark

Data pipelines

Databricks

Kafka

RAG

CI/CD

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Databricks

Kafka

Delta Lake

Microsoft Fabric

Job description

Morgan McKinley Pte Ltd is seeking a Senior Data Engineer to architect, develop, and refine high-throughput data systems powering analytics, AI, and RAG solutions.

You will build scalable ingestion/processing across hybrid cloud, optimise batch/streaming workflows with Databricks and Kafka, and deliver secure data for reporting, ML, and GenAI use cases. Strong Python/SQL and Spark experience required.