Senior AI Data Engineer #AIDA (Singapore, Singapore)

Singtel; Singapore; On-site; Confidential; Full time

Job summary

Singtel is expanding its AI and Data Analytics unit (AIDA) and seeks a data platform engineer to design, build, and operate scalable data pipelines. You will optimize batch and streaming workloads using Databricks and Kafka, deliver high-quality data for analytics and ML, and implement knowledge base and RAG solutions while upholding governance and security standards.

You will collaborate with engineers, architects, and product owners to translate requirements into robust, maintainable platform

Qualifications

• Bachelor’s degree in CS, Engineering, or related field.
• 5–8 years in data engineering or cloud analytics delivery with production pipelines.
• Proven ability to design and operate production batch/streaming data pipelines.
• Hands-on Python and SQL for transformation and validation.
• Experience with PySpark and Spark for large-scale processing.
• Knowledge base and RAG solutions for agentic AI use cases.
• Strong problem-solving, communication, and documentation skills.
• Understand enterprise data architecture, security, CI/CD, and production ops.

Responsibilities

• Design, build, operate scalable data ingestion, transformation, and serving on a modern data platform.
• Develop and optimise batch and streaming pipelines using Databricks and Kafka.
• Deliver reliable data for analytics, reporting, and ML use cases.
• Lead integration of data from diverse sources and ensure governance and security.
• Contribute to automation, CI/CD, and knowledge base / RAG solutions.
• Ensure production readiness, observability, and incident response for platform components.

Skills

Data pipelines

Python

SQL

PySpark

Apache Spark

Databricks

Kafka

Knowledge base

RAG

CI/CD

Data governance

Security

Education

Bachelor’s degree in Computer Science or Engineering

Tools

Databricks

Kafka

Microsoft Fabric

Delta Lake

Job description

An empowering career at Singtel begins with a Hello. Our purpose, to Empower Every Generation, connects people to the possibilities they need to excel. Every "hello" at Singtel opens doors to new initiatives, growth, and BIG possibilities that takes your career to new heights. So, when you say hello to us, you are really empowered to say…“Hello BIG Possibilities”.

Powering the Future with AIDA

To lead the next phase of our AI evolution, we've launched a new business unit AIDA – Artificial Intelligence & Data Analytics – a strategic engine driving our transformation designed to scale our AI ambitions with precision and purpose.This marks a pivotal shift in how we operate, innovate, and serve to embed intelligence into every layer of our business.

At Singtel, this is more than a technology upgrade. It’s a strategic transformation that redefines how value is created across the enterprise core— augmenting human capabilities and unlocking entirely new potential. It is a transformation journey by aligning people, platforms, and processes under one cohesive strategy. Our mission is to build AI literacy, and foster a culture where intelligence empowers people.

We welcome you to join us

on a transformational journey that’s reshaping the telecommunications industry — and redefining what’s possible with AI at its core. Grow with us in a workplace that champions innovation, embraces agility, and puts human potential at the heart of everything we do.

Be a Part of Something BIG!

• Responsible for designing, building, and operating scalable data ingestion, transformation, and serving capabilities across a modern hybrid cloud data platform, ensuring solutions are reliable, secure, reusable, and aligned to enterprise architecture and governance standards.
• Develop and optimise batch and streaming pipelines using cloud tools such as Databricks and Kafka, applying sound engineering practices to ensure performance, resilience, and maintainability.
• Contribute to the delivery of reliable, secure, and high-quality data for analytics, reporting, and machine learning use cases
• Lead the implementation and operationalisation of knowledge base and retrieval-augmented generation solution stacks to support scalable GenAI and agentic use cases across business domains

Make an Impact by:

• Design, build, optimise, and maintain batch and streaming data ingestion pipelines using platforms such as Databricks and Kafka, ensuring scalability, reliability, observability, and alignment with enterprise data architecture standards.
• Perform data transformation and cleansing using PySpark or SQL based on business and technical requirements
• Monitor and troubleshoot data workflows to ensure data quality and pipeline reliability
• Provide technical guidance to engineers and delivery partners on data platform patterns, reusable components, code quality, deployment readiness, and production support practices.
• Lead integration of data from diverse source systems including files, APIs, databases, and streaming platforms, working with source-system owners and consuming teams to define fit-for-purpose ingestion patterns and delivery timelines.
• Help maintain metadata and pipeline documentation for transparency and traceability
• Own production readiness for assigned data and AI platform components, including observability, incident triage, root-cause analysis, release coordination, and continuous improvement of operational runbooks.
• Participate in integrating pipelines with tools such as Microsoft Fabric, Databricks, Delta Lake, and other platform components
• Build and maintain knowledge base and RAG solution on variety of hosting platforms
• Implement and operate knowledge base storage, lifecycle management and embedding/vectorization
• Contribute to automation efforts using version control and CI/CD workflows
• Apply data governance, security, access control, and operational risk policies during solution design and implementation, ensuring pipelines and knowledge platforms meet enterprise compliance requirements.

Skills for Success:

• Bachelor’s degree in Computer Science, Engineering, or a related field
• 5–8 years of experience in data engineering, data platform engineering, or cloud-scale analytics solution delivery, with demonstrated ownership of production pipelines and platform components.
• Proven ability to independently design, build, optimise, and operate production‑grade batch or streaming data pipelines, including orchestration, observability, error handling, performance tuning, and operational support.
• Hands‑on experience with Python and SQL for data transformation and validation
• Familiarity with Apache Spark (especially PySpark) and large‑scale data processing concepts
• Experience with implementing knowledge base and RAG solutions for agentic AI use cases
• Self‑starter with strong problem‑solving skills and a keen attention to detail
• Able to work independently and lead technical discussions with engineers, architects, product owners, source‑system teams, and business stakeholders to translate requirements into secure and maintainable platform solutions.
• Strong documentation and communication skills
• Strong understanding of enterprise data architecture, cloud security, access control, CI/CD, release management, and production operations for data and AI platform solutions.