Senior Big Data & AI Analytics Lead

OX CONSULTANCY PTE. LTD.; Singapore; On-site; SGD 180,000 - 260,000; Full time

Job summary

OX CONSULTANCY PTE. LTD. in Singapore seeks an experienced Big Data Engineering Lead to design, develop and lead scalable enterprise data platforms, real-time data pipelines and analytics solutions.

The successful candidate will work with large-volume datasets, cloud platforms, distributed processing technologies and modern AI/GenAI solutions. You will lead the development of batch and real-time data pipelines, design data models and data warehouses, and mentor data engineering teams while

Qualifications

• 8+ years of experience in Data Engineering / Big Data / Data Analytics.
• Strong hands-on experience with Python, SQL, PySpark, Apache Spark and Kafka.
• Experience in AWS cloud data services, including S3 and related data engineering technologies.
• Strong experience with Airflow or equivalent workflow orchestration tools.
• Experience with Snowflake, Hadoop/HDFS, Hive or other enterprise data platforms.
• Ability to design ETL, data warehousing, data modelling and distributed data processing.
• Experience developing REST APIs using FastAPI or Flask.
• Exposure to GenAI, LLM, RAG, LangChain/LangGraph or Agentic AI is an advantage.
• Strong analytical, problem-solving and technical leadership skills.
• Experience working in Agile software development environments.

Responsibilities

• Lead the design and development of scalable Big Data Engineering and Data Analytics platforms.
• Design and implement batch and real-time data pipelines using Python, PySpark, Spark Structured Streaming and Kafka.
• Develop data ingestion, transformation and processing workflows using AWS, S3, Airflow and orchestration tools.
• Design data models, data warehouses, data lakes/lakehouse architectures and high-performance analytics solutions.
• Develop and optimize complex SQL queries, ETL processes and data processing workflows.
• Build REST APIs and backend services using FastAPI/Flask for data and analytics applications.
• Implement data replication, archival, reconciliation, quality monitoring and governance processes.
• Deploy scalable data applications on AWS, including containerized environments and CI/CD pipelines.
• Lead development of GenAI/LLM, RAG and Agentic AI solutions for enterprise analytics and natural-language data querying.
• Provide technical leadership, mentoring and guidance to data engineering teams.
• Collaborate with business and technology stakeholders to translate requirements into scalable data solutions.

Skills

Python

SQL

Distributed data processing

Big Data / Data Analytics

Leadership / mentoring

Agile

Education

Degree in Computer Science or related

Tools

PySpark

Apache Spark

Kafka

Airflow

AWS

S3

Snowflake

Hadoop/HDFS

Hive

REST APIs

FastAPI/Flask

GenAI/LLM

LangChain/LangGraph

Kubernetes

CI/CD

Job description

OX CONSULTANCY PTE. LTD. in Singapore seeks an experienced Big Data Engineering Lead to design, develop and lead scalable enterprise data platforms, real-time data pipelines and analytics solutions.

The successful candidate will work with large-volume datasets, cloud platforms, distributed processing technologies and modern AI/GenAI solutions. You will lead the development of batch and real-time data pipelines, design data models and data warehouses, and mentor data engineering teams while