Senior AI Data Engineer: Scalable GenAI Pipelines

Singtel; Singapore; On-site; Confidential; Full time

Job summary

Singtel is expanding its AI and Data Analytics unit (AIDA) and seeks a data platform engineer to design, build, and operate scalable data pipelines. You will optimize batch and streaming workloads using Databricks and Kafka, deliver high-quality data for analytics and ML, and implement knowledge base and RAG solutions while upholding governance and security standards.

You will collaborate with engineers, architects, and product owners to translate requirements into robust, maintainable platform

Qualifications

• Bachelor’s degree in CS, Engineering, or related field.
• 5–8 years in data engineering or cloud analytics delivery with production pipelines.
• Proven ability to design and operate production batch/streaming data pipelines.
• Hands-on Python and SQL for transformation and validation.
• Experience with PySpark and Spark for large-scale processing.
• Knowledge base and RAG solutions for agentic AI use cases.
• Strong problem-solving, communication, and documentation skills.
• Understand enterprise data architecture, security, CI/CD, and production ops.

Responsibilities

• Design, build, operate scalable data ingestion, transformation, and serving on a modern data platform.
• Develop and optimise batch and streaming pipelines using Databricks and Kafka.
• Deliver reliable data for analytics, reporting, and ML use cases.
• Lead integration of data from diverse sources and ensure governance and security.
• Contribute to automation, CI/CD, and knowledge base / RAG solutions.
• Ensure production readiness, observability, and incident response for platform components.

Skills

Data pipelines

Python

SQL

PySpark

Apache Spark

Databricks

Kafka

Knowledge base

RAG

CI/CD

Data governance

Security

Education

Bachelor’s degree in Computer Science or Engineering

Tools

Databricks

Kafka

Microsoft Fabric

Delta Lake

Job description

Singtel is expanding its AI and Data Analytics unit (AIDA) and seeks a data platform engineer to design, build, and operate scalable data pipelines. You will optimize batch and streaming workloads using Databricks and Kafka, deliver high-quality data for analytics and ML, and implement knowledge base and RAG solutions while upholding governance and security standards.

You will collaborate with engineers, architects, and product owners to translate requirements into robust, maintainable platform