Senior Data Engineer (GCP + AWS)

PALO IT; Singapore; On-site; SGD 60,000 - 80,000; Full time

Job summary

A global technology consultancy is seeking a Senior Data Engineer to design and implement cloud-native data pipelines. Ideal candidates will have strong expertise in AWS and GCP, with at least 7 years of experience in data engineering. Key responsibilities include building scalable ETL/ELT pipelines, migrating workloads to the cloud, and partnering with Data Scientists to enable advanced analytics use cases. This position offers a dynamic work environment in Singapore.

Qualifications

• 7+ years in data engineering with end-to-end pipeline design and implementation.
• Strong expertise in AWS technologies such as S3, Glue, Lambda, etc.
• Strong expertise in GCP technologies such as BigQuery, Dataflow, etc.

Responsibilities

• Design and implement scalable ETL/ELT pipelines on AWS and GCP.
• Migrate existing on-premises workflows to cloud-native services.
• Build reusable data frameworks for ingestion and quality monitoring.

Skills

Data engineering expertise

AWS proficiency

GCP proficiency

Python programming

SQL development

Orchestration tools (Airflow, Cloud Composer)

Shell scripting

Tools

AWS tools

GCP tools

Airflow

Docker

Kubernetes

Job description

Overview

PALO IT is a global technology consultancy that crafts tech as a force for good. We design, develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people, planet, profit. We do the right thing, and we do it right. We are proud to be a World Economic Forum New Champion, and a B Corp-certified company.

We are small enough to care locally, big enough to deliver globally (5 continents, 18 offices, +650 experts from +50 nationalities); robust and resilient (100% independent and 0 debt); entrepreneurs and passionate experts; positive, courageous, caring, doers and committed to excellence.

Role

The projects involve building and scaling cloud-native data pipelines to handle ingestion, transformation, quality assurance, enrichment, and advanced analytics use cases across multiple industries. We are seeking a Senior Data Engineer to modernize and enhance data platforms, with strong expertise in AWS and GCP.

Responsibilities

• Design and implement scalable ETL/ELT pipelines on AWS and GCP.
• Migrate existing on-premises and hybrid workloads into cloud-native services.
• Build reusable data frameworks for ingestion, quality monitoring, and transformations.
• Implement robust error handling, recovery mechanisms, and observability into pipelines.
• Partner with Data Scientists and Architects to enable predictive/GenAI use cases.

Must Have Experience

• 7+ years in data engineering with end-to-end pipeline design and implementation.
• AWS expertise: S3, Glue, Lambda, Step Functions, EMR, Athena, Fargate, AWS Batch.
• GCP expertise: BigQuery, Dataflow, Pub/Sub, Dataproc, Cloud Functions.
• Strong Python and SQL development skills.
• Proficiency with orchestration tools like Airflow or Cloud Composer.
• Shell scripting (Linux).
• Familiarity with data formats (Parquet, ORC, JSON, Avro) and modern storage patterns.

Good to Have

• Experience integrating AWS GCP cross-cloud pipelines.
• Knowledge of containerization (Docker, Kubernetes, GKE/EKS).
• Domain knowledge in Financial Services, Healthcare, Maritime, or Retail.

Equal Opportunity

PALO IT Singapore is an equal opportunity employer. Employment decisions will be based on merit, qualifications and abilities. Palo IT SG does not discriminate in employment opportunities or practices on the basis of race, colour, religion, sex, sexuality, national origin, age, disability, marital status or any other characteristics protected by law.