AWS Data Engineer

TESCOM (SINGAPORE) SOFTWARE SYSTEMS TESTING PTE LTD.; Singapore; On-site; SGD 90,000 - 180,000; Full time

Job summary

TESCOM (SINGAPORE) SOFTWARE SYSTEMS TESTING PTE LTD. is seeking an experienced data architect/engineer to design and implement an end-to-end AWS Data Lake and Lakehouse solution in Singapore.

You will define standards, govern metadata, and build scalable ingestion pipelines using AWS services. The role requires hands-on experience with AWS data tools, SQL, and Python or Scala, plus AWS certifications or equivalent.

Qualifications

• 3–5 years in data engineering, data architecture, or cloud infrastructure roles.
• Hands-on with core AWS data services: S3, Glue, Redshift, Lambda, Kinesis, Step Functions, EventBridge, AppFlow, Lake Formation.
• Strong SQL and at least one scripting language (Python or Scala).
• Experience designing and implementing Data Lake or Lakehouse architectures.
• Solid understanding of data governance, cataloguing, and metadata management.
• AWS Certifications (Data Engineer – Associate or Solutions Architect) or equivalent.

Responsibilities

• Design end-to-end AWS Data Lake and Lakehouse architecture across Landing, Transformed and Curated zones.
• Define and govern data architecture standards, patterns, and best practices across the platform.
• Architect reusable data ingestion pipelines supporting REST APIs, JDBC, S3 uploads, and Salesforce via AWS AppFlow.
• Develop and deploy data ingestion pipelines using AWS Glue, Lambda, Step Functions, EventBridge, and API Gateway.
• Build and maintain data transformation workflows (batch and stream) using Glue and Redshift.
• Implement orchestration, monitoring, logging, and notification frameworks for pipelines.
• Ensure data security with Lake Formation, IAM, and Secrets Manager, and enforce access controls.
• Monitor platform health, cost efficiency, and maintain runbooks and documentation.

Skills

Data engineering

Data architecture

Cloud infrastructure

SQL

Python/Scala

Education

AWS Certification – Data Engineer or Solutions Architect

Tools

Amazon S3

AWS Glue

Amazon Redshift

AWS Lambda

Amazon Kinesis

AWS Step Functions

Amazon EventBridge

AWS AppFlow

AWS Lake Formation

Tableau

Job description

Key Responsibilities

Architecture & Design

• Design and architect the end-to-end AWS Data Lake and Lakehouse solution, including Landing Zone, Transformed Zone, and Curated/Consumption Zone layers
• Define and govern data architecture standards, patterns, and best practicesacross the platform
• Architect reusable data ingestion pipelines supporting REST APIs, JDBC databases, S3 file uploads, and SaaS connectors (e.g. Salesforce via AWS AppFlow)
• Design data storage strategies including hot, warm, and cold storage tiers, encryption, and data lifecycle policies

Development & Deployment

• Develop and deploy data ingestion pipelines using AWS Glue, Lambda, Step Functions, EventBridge, and API Gateway
• Build and maintain data transformation workflows (batch and stream processing) using AWS Glue and Amazon Redshift
• Implement orchestration, monitoring, logging, and notification frameworks for pipeline operations
• Develop and maintain the AWS Glue Data Catalogue, including schema evolution tracking and metadata tagging

Security & Governance

• Configure and enforce data security policies using AWS Lake Formation, IAM, and Secrets Manager
• Implement granular access controls at database, table, and column levels
• Ensure compliance with data classification, retention, and audit requirements
• Support data quality frameworks and observability monitoring

Maintenance & Operations

• Monitor platform health, performance, and pipeline reliability
• Troubleshoot and resolve data pipeline failures and data quality issues
• Maintain documentation for architecture decisions, pipeline configurations, and operational runbooks
• Continuously optimise platform performance and cost efficiency on AWS

Requirements

Essential

• Minimum 3 to 5 years of experience in data engineering, data architecture, or cloud infrastructure roles
• Hands-on expertise with core AWS data services: Amazon S3, AWS Glue, Amazon Redshift, AWS Lambda, Amazon Kinesis, AWS Step Functions, Amazon EventBridge, AWS AppFlow, AWS Lake Formation
• Strong proficiency in SQL and at least one scripting language (Python or Scala)
• Experience designing and implementing Data Lake or Lakehouse architectures
• Solid understanding of data governance, data cataloguing, and metadata management
• Experience with batch and streaming data processing patterns
• AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect certification (or equivalent)

Preferred

• Experience integrating with Tableau or similar BI visualisation tools via Amazon Redshift or S3
• Familiarity with MLOps frameworks and AI/ML model deployment on AWS SageMaker
• Experience with Salesforce data integration using AWS AppFlow
• Knowledge of Change Data Capture (CDC) and incremental data load patterns
• Prior experience in a government or public sector data environment