Data Engineer

QUESS SELECTION & SERVICES PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

QUESS SELECTION & SERVICES PTE. LTD. is seeking a data platform engineer to design, build, and operate scalable data pipelines across AWS cloud and hybrid environments in Singapore.

You will implement ETL/ELT workflows with Python, PySpark, SQL, and cloud-native tools, develop lakehouse architectures, and ensure reliability, security, and cost controls. The role emphasizes collaboration with cross-functional teams to modernize platforms, automate processes, and deliver production-grade data

Qualifications

• 3–5+ years of hands-on experience developing and supporting production-grade data pipelines using SQL, Python, and cloud technologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines and platform solutions across AWS cloud and hybrid/on-premises environments.
• Build and optimize ETL/ELT workflows using Python, PySpark, SQL, and cloud-native technologies.
• Develop and support data lakes, lakehouse architectures, metadata management, search platforms, and AI-enabled services.
• Implement and maintain CI/CD pipelines, automated testing frameworks, Infrastructure as Code (IaC), and deployment processes.
• Integrate and support AWS services and platform technologies, including OpenSearch, Amazon Bedrock, AgentCore, and Model Context Protocol (MCP).
• Monitor and optimize platform reliability, security, performance, observability, and cloud costs.
• Troubleshoot production issues, perform root cause analysis, and ensure platform stability and operational excellence.
• Collaborate with cross-functional teams to drive platform modernization, automation, and continuous improvement.

Skills

SQL

Python

PySpark

Apache Spark

Data engineering

Distributed processing

Tools

AWS

OpenSearch

Amazon Bedrock

AgentCore

MCP

Apache Airflow

dbt

Kafka/MSK

Iceberg

Hive

Trino

Presto

Git

CI/CD

IaC

Job description

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and platform solutions across AWS cloud and hybrid/on-premises environments.

• Build and optimize ETL/ELT workflows using Python, PySpark, SQL, and cloud-native technologies.

• Develop and support data lakes, lakehouse architectures, metadata management, search platforms, and AI-enabled services.

• Implement and maintain CI/CD pipelines, automated testing frameworks, Infrastructure as Code (IaC), and deployment processes.

• Integrate and support AWS services and platform technologies, including OpenSearch, Amazon Bedrock, AgentCore, and Model Context Protocol (MCP).

• Monitor and optimize platform reliability, security, performance, observability, and cloud costs.

• Troubleshoot production issues, perform root cause analysis, and ensure platform stability and operational excellence.

• Collaborate with cross-functional teams to drive platform modernization, automation, and continuous improvement.

Mandatory Requirements

• 3–5+ years of hands-on experience developing and supporting production-grade data pipelines using SQL, Python, and cloud technologies.

• Practical experience with AWS cloud services and/or hybrid cloud environments.

• Strong Linux/Unix administration, shell scripting, and production support capabilities.

• Solid understanding of data engineering principles, distributed processing, and enterprise platform operations.

Technical Skills

• Data Engineering: ETL/ELT, data lakes, lakehouse architecture, distributed data processing, SQL, Python, PySpark, and Apache Spark.

• AWS & Cloud Platforms: AWS Glue, EMR, S3, Redshift, Lambda, and Amazon Bedrock.

• Data & Streaming Technologies: Apache Airflow, dbt, Kafka/MSK, Apache Iceberg, Hive, Trino, and Presto.

• Search & AI Platforms: OpenSearch, AgentCore, MCP, and AI service integration.

• DevOps & Platform Engineering: Git, CI/CD, DevOps practices, Infrastructure as Code (IaC), and automated testing.

• Monitoring & Observability: CloudWatch, Datadog, Grafana, and Prometheus.

Preferred Qualifications

• Experience building and supporting data lake and lakehouse platforms.

• Knowledge of streaming pipelines and event-driven architectures.

• Hands‑on experience with OpenSearch administration, performance optimization, and tuning.

• Familiarity with monitoring, logging, alerting, and observability tools.

• Understanding of data governance, security, access controls, and Personally Identifiable Information (PII) protection.

• Experience supporting enterprise platforms within banking, insurance, or financial services environments.