Senior Data & AI Solutions Engineer

UNISONEDGE CONSULTING PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

UNISONEDGE CONSULTING PTE. LTD. is seeking a senior data engineer to design, develop, and optimize scalable data engineering solutions using Big Data, cloud, and AI technologies.

You will build high-performance data pipelines, ingestion frameworks, and processing solutions while collaborating with architects, data scientists, and business stakeholders to deliver production-ready systems.

Qualifications

• Bachelor's or master's in computer science, IT, engineering, or related field.
• 8+ years in data engineering/big data with large-scale processing.
• Strong programming in Python, Java, or Scala.
• Hands-on Spark/PySpark, SQL, Hadoop, and Hive experience.
• Experience with cloud platforms, especially AWS.
• Experience with data warehouses, relational DBs, and data lake tech.
• Experience with Kafka, Airflow, Jenkins, Git and CI/CD.
• Docker, Kubernetes or OpenShift is advantageous.
• Knowledge of Databricks, Snowflake, Cloudera is advantageous.
• Exposure to generative AI, LLMs, RAG, LangChain, vector DBs is an advantage.

Responsibilities

• Design, develop, and maintain scalable batch and real-time data pipelines.
• Develop and optimize ETL/ELT pipelines for large-scale processing.
• Build data ingestion solutions using databases, APIs, files, and streaming platforms.
• Work with AWS services (S3, Glue, EMR, Redshift, Kinesis, Lambda, DynamoDB).
• Develop and optimize data processing using Hadoop, Hive, Spark, Kafka, Cloudera, Databricks.
• Perform SQL and Spark performance tuning for large workloads.
• Design data models, warehouses, and lake solutions.
• Develop CI/CD pipelines and support containerized deployments (Docker, Kubernetes, OpenShift).
• Integrate generative AI and NLP into enterprise data apps as needed.
• Develop solutions with LLM frameworks, RAG, vector databases, and AI APIs.
• Collaborate with solution architects, data scientists, engineers, and stakeholders to deliver production-grade solutions.
• Participate in design, development, testing, deployment and production support.

Skills

Python

Java

Scala

SQL

Spark/PySpark

Hadoop

Hive

Kafka

Airflow

CI/CD

Docker

Kubernetes

OpenShift

Databricks

Snowflake

Cloudera

AWS

Education

Bachelor's degree in CS/IT/Engineering or related field

Master's degree in CS/IT/Engineering or related field

Tools

Apache Spark/PySpark

Hadoop

Hive

Databricks

Cloudera

Snowflake

Docker

Kubernetes

OpenShift

Airflow

Jenkins

Git

AWS

S3

Glue

EMR

Redshift

Kinesis

Job description

UNISONEDGE CONSULTING PTE. LTD. is seeking a senior data engineer to design, develop, and optimize scalable data engineering solutions using Big Data, cloud, and AI technologies.

You will build high-performance data pipelines, ingestion frameworks, and processing solutions while collaborating with architects, data scientists, and business stakeholders to deliver production-ready systems.