Senior Data Engineer - AI & Big Data Platforms

JEET ANALYTICS PTE. LTD.; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

JEET ANALYTICS PTE. LTD. is seeking a highly skilled Senior Data Engineer to design, develop, and optimize scalable data pipelines using Big Data and cloud technologies. You will work with architects, data scientists, and stakeholders to deliver production-grade solutions.

The role demands 8+ years of data engineering experience, strong Python/Java/Scala skills, and hands-on Spark/Hadoop/Hive. Experience with AWS and data lake/warehouse technologies is essential.

Qualifications

• Bachelor’s or master's degree in computer science, information technology, engineering, or a related field.
• Minimum 8 years of relevant experience in data engineering/big data.
• Strong programming experience in Python, Java, and/or Scala.
• Hands-on with Apache Spark/PySpark, SQL, Hadoop, and Hive.
• Experience with cloud platforms, particularly AWS.
• Experience with data warehouses, relational databases, and data lake technologies.
• Experience with Kafka, Airflow, Jenkins, Git, and CI/CD practices.
• Experience with Docker, Kubernetes, or OpenShift is advantageous.
• Knowledge of Databricks, Snowflake, and Cloudera is advantageous.
• Exposure to generative AI, LLMs, RAG, LangChain/LangGraph, and vector databases is an advantage.

Responsibilities

• Design, develop, and maintain scalable batch and real-time data pipelines using Spark/PySpark, SQL, and Python.
• Develop and optimize ETL/ELT pipelines for large-scale data processing.
• Build data ingestion solutions using databases, APIs, files, and streaming platforms.
• Work with AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda, and DynamoDB.
• Develop and optimize data processing solutions using Hadoop, Hive, Spark, Kafka, Cloudera, and Databricks.
• Perform SQL and Spark performance tuning and optimize large-scale data processing workloads.
• Design and implement data models, data warehouses, and data lake solutions.
• Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes, and OpenShift.
• Integrate generative AI and NLP capabilities into enterprise data applications where required.
• Collaborate with solution architects, data scientists, software engineers, and business stakeholders to deliver production-ready solutions.
• Participate in system design, development, testing, deployment, and production support.

Skills

Python

Java

Scala

PySpark

SQL

Apache Spark

Hadoop

Hive

Kafka

Databricks

Cloudera

AWS

S3

Glue

EMR

Redshift

Kinesis

Docker

Kubernetes

OpenShift

Snowflake

MongoDB

Oracle

PostgreSQL

Generative AI

LLMs

LangChain

Education

Bachelor’s or Master’s in Computer Science / IT / Engineering

Tools

Apache Spark

Hadoop

Hive

Kafka

Databricks

Cloudera

AWS Suite

OpenShift

Docker

Kubernetes

Airflow

Jenkins

Git

Snowflake

MongoDB

Oracle

PostgreSQL

Job description

JEET ANALYTICS PTE. LTD. is seeking a highly skilled Senior Data Engineer to design, develop, and optimize scalable data pipelines using Big Data and cloud technologies. You will work with architects, data scientists, and stakeholders to deliver production-grade solutions.

The role demands 8+ years of data engineering experience, strong Python/Java/Scala skills, and hands-on Spark/Hadoop/Hive. Experience with AWS and data lake/warehouse technologies is essential.