Senior Data & AI Solutions Engineer

RISKDATA CONSULTING PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

RiskData Consulting Pte. Ltd. in Singapore seeks a Senior Data Engineer to design, develop, and optimize scalable data engineering solutions using Big Data, cloud, and AI technologies.

You will build high-performance data pipelines, ingestion frameworks, and processing solutions in collaboration with architects, data scientists and business stakeholders. The role requires 8+ years in Data Engineering with strong Spark, Hadoop, Kafka and AWS experience, plus CI/CD and containerization expertise.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering or a related field.
• Minimum 8 years of relevant experience in Data Engineering / Big Data, with strong experience in large-scale data processing.
• Strong programming experience in Python, Java and/or Scala.
• Strong hands-on experience with Apache Spark/PySpark, SQL, Hadoop and Hive.
• Experience with cloud platforms, particularly AWS.
• Experience with data warehouses, relational databases and data lake technologies.
• Experience with Kafka, Airflow, Jenkins, Git and CI/CD practices.
• Experience with Docker, Kubernetes or OpenShift is advantageous.
• Knowledge of Databricks, Snowflake and Cloudera is advantageous.
• Exposure to Generative AI, LLMs, RAG, LangChain/LangGraph and vector databases is an advantage.

Responsibilities

• Design, develop and maintain scalable batch and real-time data pipelines using Apache Spark/PySpark, SQL and Python.
• Develop and optimize ETL/ELT pipelines for large-scale data processing.
• Build data ingestion solutions using databases, APIs, files and streaming platforms.
• Work with AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda and DynamoDB.
• Develop and optimize data processing solutions using Hadoop, Hive, Spark, Kafka, Cloudera and Databricks.
• Perform SQL and Spark performance tuning and optimize large-scale data processing workloads.
• Design and implement data models, data warehouses and data lake solutions.
• Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes and OpenShift.
• Integrate Generative AI and NLP capabilities into enterprise data applications where required.
• Develop solutions using LLM frameworks, RAG, vector databases and AI APIs.
• Collaborate with solution architects, data scientists, software engineers and business stakeholders to deliver production-ready solutions.
• Participate in system design, development, testing, deployment and production support.

Skills

Python/Java/Scala

Analytical thinking

Communication

Education

Bachelor's or Master’s degree in CS/IT/Engineering

Tools

Apache Spark/PySpark

Hadoop/Hive

Kafka

Airflow/Jenkins

Docker/Kubernetes/OpenShift

Databricks/Snowflake/Cloudera

AWS (S3/Glue/EMR/Redshift)

Job description

RiskData Consulting Pte. Ltd. in Singapore seeks a Senior Data Engineer to design, develop, and optimize scalable data engineering solutions using Big Data, cloud, and AI technologies.

You will build high-performance data pipelines, ingestion frameworks, and processing solutions in collaboration with architects, data scientists and business stakeholders. The role requires 8+ years in Data Engineering with strong Spark, Hadoop, Kafka and AWS experience, plus CI/CD and containerization expertise.