Senior Data Engineer: Scalable Pipelines & AI-Driven Data

JEET ANALYTICS PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

JEET ANALYTICS PTE. LTD. seeks a Senior Data Engineer to design, develop and optimize scalable data pipelines and AI-enabled data applications. You will work with architects, data scientists and engineers to deliver production-ready solutions for batch and real-time processing.

Requirements include 8+ years in data engineering, strong Python/Java/Scala, Spark/PySpark, Hadoop/Hive, and AWS expertise. Knowledge of Databricks, Snowflake and Cloudera is advantageous.

Qualifications

• Bachelor’s or Master’s degree in CS/IT/Engineering or a related field
• 8+ years of Data Engineering/Big Data experience with large-scale processing
• Strong programming in Python, Java and/or Scala
• Hands-on experience with Spark/PySpark, SQL, Hadoop and Hive
• Experience with AWS and data warehouses/data lake tech
• Experience with Kafka, Airflow, Jenkins, Git and CI/CD
• Docker, Kubernetes or OpenShift experience advantageous
• Databricks, Snowflake and Cloudera advantageous
• Exposure to Generative AI, LLMs, RAG, LangChain/LangGraph and vector databases advantageous
• Strong analytical, problem-solving and communication skills

Responsibilities

• Design, develop and maintain scalable batch and real-time data pipelines using Spark/PySpark, SQL and Python
• Develop and optimize ETL/ELT pipelines for large-scale data processing
• Build data ingestion solutions using databases, APIs, files and streaming platforms
• Work with AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda and DynamoDB
• Develop and optimize data processing using Hadoop, Hive, Spark, Kafka, Cloudera and Databricks
• Perform SQL and Spark performance tuning for large-scale workloads
• Design and implement data models, data warehouses and data lakes
• Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes and OpenShift
• Integrate Generative AI and NLP into enterprise data apps as needed
• Develop solutions using LLM frameworks, RAG, vector databases and AI APIs
• Collaborate with solution architects, data scientists, software engineers and stakeholders
• Participate in system design, development, testing, deployment and production support

Skills

Analytical skills

Communication skills

Education

Bachelor’s or Master’s degree in CS/IT/Engineering

Tools

Python

Java

Scala

Spark

PySpark

SQL

Hadoop

Hive

Kafka

Airflow

Jenkins

Git

CI/CD

Docker

Kubernetes

OpenShift

Databricks

Snowflake

Cloudera

AWS

S3

Glue

EMR

Redshift

Kinesis

Job description

JEET ANALYTICS PTE. LTD. seeks a Senior Data Engineer to design, develop and optimize scalable data pipelines and AI-enabled data applications. You will work with architects, data scientists and engineers to deliver production-ready solutions for batch and real-time processing.

Requirements include 8+ years in data engineering, strong Python/Java/Scala, Spark/PySpark, Hadoop/Hive, and AWS expertise. Knowledge of Databricks, Snowflake and Cloudera is advantageous.