AI Data Platform Engineer — Pipelines & Reliability

JOBLINE RESOURCES PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

JOBLINE RESOURCES PTE. LTD. is seeking a Data Platform Engineer in Singapore to design, build, and operate production-grade batch and streaming data pipelines using Databricks and Kafka.

You will transform data with PySpark/SQL, ensure data quality, and drive observability and governance across the platform. Key responsibilities include leading ingestion from diverse sources, mentoring engineers, and coordinating with source-system teams to enable secure, scalable solutions.

Qualifications

• Bachelor's degree or higher in CS/Engineering or related field.
• 5–8 years of data engineering or cloud-scale analytics experience with production pipelines.
• Proven ability to design, build, optimise, and operate production-grade data pipelines.
• Hands-on Python and SQL for data transformation and validation.
• Familiarity with PySpark and Spark concepts at scale.
• Experience with knowledge base and RAG solutions for AI use cases.
• Strong problem-solving, communication, and leadership in technical discussions.

Responsibilities

• Design, build, optimise, and maintain batch and streaming pipelines with Databricks and Kafka.
• Perform data transformation using PySpark or SQL.
• Monitor and troubleshoot data workflows for quality and reliability.
• Guide engineers and delivery partners on platform patterns and code quality.
• Lead ingestion from files, APIs, databases, and streaming sources with defined patterns.
• Maintain metadata and pipeline documentation for transparency.
• Own production readiness, observability, incident triage, and runbook improvements.
• Integrate pipelines with Microsoft Fabric, Databricks, Delta Lake, and other components.
• Develop and maintain knowledge base and RAG solutions across hosting platforms.
• Implement knowledge base storage, lifecycle management, and embeddings.

Skills

Python

SQL

PySpark

Apache Spark

Databricks

Kafka

CI/CD

Observability

Data governance

Communication

Education

Bachelor's degree in Computer Science/Engineering

Tools

Microsoft Fabric

Databricks

Delta Lake

Job description

JOBLINE RESOURCES PTE. LTD. is seeking a Data Platform Engineer in Singapore to design, build, and operate production-grade batch and streaming data pipelines using Databricks and Kafka.

You will transform data with PySpark/SQL, ensure data quality, and drive observability and governance across the platform. Key responsibilities include leading ingestion from diverse sources, mentoring engineers, and coordinating with source-system teams to enable secure, scalable solutions.