Data Engineer Lead

DT One group.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

DT One is seeking a data engineer lead in Singapore or Manila to design, build, and maintain a scalable data and AI platform. You will collaborate with engineering leads, data scientists, and analysts to solve operational challenges and enable organizational growth.

The role emphasizes data quality, security, and scalable ML/AI deployments, including vector databases and LLM orchestration. You will drive data governance and platform adoption across teams.

Qualifications

• Expertise in medium- to large-scale data processing systems.
• Developers with strong Python/Java/Scala skills and SQL prowess.
• Experience with AWS services and Docker environments.
• Hands-on with distributed processing (Spark, Presto, Hadoop).
• Solid background in Data Warehousing and Data Modeling.
• Familiar with MLOps, LLM orchestration, and vector databases.

Responsibilities

• Design and build a reliable data lake at scale with data quality and availability.
• Build complex data pipelines adding new sources and capabilities.
• Develop internal tools, libraries, ETL automation, and data exploration frameworks.
• Migrate ML and Data Apps from POC to production with high availability and security.
• Architect and scale infrastructure for Generative AI and vector retrieval (RAG).
• Monitor ETLs, Data Warehouses, and Data Apps.
• Provide guidance on data governance and best practices.
• Support cross-functional teams on data usage and platform adoption.

Skills

Large-scale data processing

Python/Java/Scala

SQL & DB administration

AWS & Docker

Spark/Presto/Hadoop

Data Warehousing & Modeling

MLOps / LangChain / LlamaIndex

Vector databases (Pinecone/Milvus/Qdr)

Tools

Docker

AWS

LangChain

Job description

About the role

DT One is seeking a highly driven, self-motivated Data Engineer Lead based in Singapore or Manila. Designing, developing, and maintaining a data and AI platform that works at scale. Collaborate with Engineering Leads, Engineers, Data Scientists, Business Analysts, and other teams within DTOne to deliver a high-performing system that addresses daily operational challenges and supports organizational scalability.

Key responsibilities

• Design and build a highly reliable data lake operating at scale with a strong emphasis on data quality, integrity, and availability.
• Build complex new data pipelines to continuously expand capabilities with new data sources.
• Develop tools, internal libraries, automated ETL creation, and data exploration frameworks to empower Data Scientists, Business Analysts, and Engineers.
• Migrate Machine Learning and Data Applications from POC/MVP stage to production grade (ensuring high availability, scalability, and security).
• Architect and scale infrastructure for Generative AI, LLM deployment, model fine-tuning, and vector retrieval (RAG).
• Monitor and maintain all data systems, including ETLs, Data Warehouses, and Data Applications.
• Provide actionable insights on Data Governance, industry best practices, and large-scale processing.
• Support cross-functional teams with guidance on data usage, processing, and platform adoption.

Qualifications and Experience

• Expertise in medium- to large-scale data processing systems, preferably based on open-source systems or AWS services.
• Strong software development skills in one or more languages such as Python, Java, or Scala.
• Advanced knowledge of SQL and SQL Database administration.
• Hands-on experience with AWS infrastructure and Docker frameworks.
• Experience with distributed processing systems such as Spark, Presto, or Hadoop.
• Solid background in Data Warehousing and Data Modeling.
• Experience with MLOps frameworks, LLM orchestration tools (e.g., LangChain, LlamaIndex), and vector databases (e.g., Pinecone, Milvus, Qdrant).

The ideal candidate is someone who:

• Has a bias towards action and shipping, starting small and growing iteratively.
• Is obsessed with data compliance, security, quality, and integrity.
• Is passionate about learning new technologies, languages, infrastructure, and data analysis to drive meaningful impact.
• Stays creative and generates fresh ideas leveraging technical expertise.
• Enjoys sharing knowledge through coaching, events, and collaboration.