Senior Data Engineer — AI-Driven Data Platforms

JEET ANALYTICS PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

JEET ANALYTICS PTE. LTD. is seeking a senior data engineer to design, develop and maintain scalable ETL/ELT data pipelines for large-volume datasets, both structured and unstructured.

You will work with Python, PySpark, Spark, Scala, Kafka, AWS Glue and Airflow to build batch and real-time data processing platforms, integrate with various databases and cloud services, and optimise performance across the enterprise.

Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• Experience with Big Data, Cloud Computing, or related tech domains.
• Experience with large-scale enterprise data platforms and production data pipelines.
• Strong programming and SQL skills.
• Experience with cloud-based data engineering and modern data processing frameworks.
• Experience with AI/ML, NLP or Generative AI is advantageous.

Responsibilities

• Design, develop and maintain scalable ETL/ELT data pipelines for large datasets.
• Develop high-performance data processing solutions using Python, PySpark, Spark, Scala.
• Build batch and real-time data pipelines using Kafka, Kinesis, Spark Streaming, Glue and Airflow.
• Develop ingestion frameworks integrating RDBMS, APIs, files, cloud storage and streaming platforms.
• Design and implement data lakes, data warehouses and cloud-based data processing platforms.
• Work with Databricks, Hadoop, Hive, Cloudera, Presto and Snowflake for large-scale processing.
• Perform data modelling, data transformation, data quality, query optimisation and performance tuning.
• Develop and optimise SQL solutions across Oracle, SQL Server, PostgreSQL, Teradata, MongoDB and cloud DBs.
• Design and implement data migration solutions involving large-scale datasets.
• Develop and support real-time and batch processing architectures for enterprise apps.
• Integrate data platforms with REST APIs, GraphQL and enterprise applications.
• Implement CI/CD and DevOps with Jenkins, Git, Docker, Kubernetes and OpenShift.
• Develop cloud-native data solutions using AWS and Azure, including S3, Glue, EMR, Redshift, Kinesis, Lambda, RDS, DynamoDB.
• Develop AI/GenAI-enabled data solutions involving LLMs, NLP, RAG, Agentic AI and vector databases.
• Integrate LLM services and AI platforms such as Azure OpenAI, OpenAI APIs, Hugging Face and Google Gemini/ADK.
• Develop NLP pipelines for text processing, embeddings, summarisation, sentiment analysis, voice-to-text and speaker diarisation.
• Design and implement vector search and retrieval solutions using Redis, ChromaDB and FAISS.
• Develop AI-powered APIs and applications using FastAPI, Gradio and Python.
• Collaborate with architects, data scientists, software engineers, business analysts and product teams to deliver enterprise data solutions.
• Participate in Agile SDLC activities including requirements analysis, architecture, development, testing, deployment and production support.
• Troubleshoot complex data, application and platform issues and provide scalable technical solutions.

Skills

ETL/ELT data pipelines

Python

PySpark

Apache Spark

Spark SQL

Scala

Kafka

Kinesis

Spark Streaming

Airflow

RDBMS

APIs

cloud storage

Databricks

Hadoop

Hive

Cloudera

Snowflake

SQL

Oracle

PostgreSQL

Teradata

MongoDB

REST APIs

GraphQL

Jenkins

Docker

Kubernetes

OpenShift

AWS

Azure

LLMs

NLP

Vector databases

Redis

FAISS

FastAPI

Education

Bachelor's or Master's degree in Computer Science, IT, Programming & Systems Analysis, Computer Studies or related

Tools

Databricks

Hadoop

Hive

Cloudera

Snowflake

Spark

Job description

JEET ANALYTICS PTE. LTD. is seeking a senior data engineer to design, develop and maintain scalable ETL/ELT data pipelines for large-volume datasets, both structured and unstructured.

You will work with Python, PySpark, Spark, Scala, Kafka, AWS Glue and Airflow to build batch and real-time data processing platforms, integrate with various databases and cloud services, and optimise performance across the enterprise.