Cloud Data Engineer — Real-Time Pipelines & Data Warehouses

Visa Hunt; Singapore; On-site; SGD 90,000 - 140,000; Full time

Job summary

Visa Hunt is seeking a data engineer to design, develop and deploy data tables, views and data marts across warehouses and data lakes. You will implement end-to-end data flows and contribute to data virtualization initiatives.

You will build scalable batch and real-time pipelines using SQL, Python and AWS services, ensuring data quality, governance and reliable integration across platforms. Collaborating with product, analytics and engineering teams, you will own APIs, data models and CI/CD

Qualifications

• Proficient in data cleaning and transformation (SQL, pandas, R) for data accuracy.
• Experience building ETL pipelines (SSIS, AWS DMS, Python, AWS Lambda, ECS, AWS Glue).
• Proficient in database design across SQL, PostgreSQL, MongoDB, Cassandra, and more.
• Experience with cloud platforms (AWS, Azure, Google Cloud).
• Experience building production-grade data pipelines and ETL/ELT integrations.
• Knowledge of system design, data structures and algorithms.
• Familiar with data modelling, Data Lake/Datastore concepts and data governance.
• Familiar with REST APIs and web requests.
• Familiar with Hadoop, Spark, Kafka, RabbitMQ.
• Familiar with web scraping tools (BeautifulSoup, CasperJS, PhantomJS, Selenium, Node.js).
• Familiar with data governance policies and security best practices.
• Comfortable in scripting languages (SQL, Python).
• Comfortable in Windows and Linux development environments.
• Interested in bridging engineering and analytics.

Responsibilities

• Design, develop and deploy data tables, views and marts in data warehouses, operational data store, data lake and data virtualization.
• Perform data extraction, cleaning, transformation, and flow. Web scraping may be also a part of the work scope in data extraction.
• Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks.
• Integrate and collate data silos in a manner which is both scalable and compliant.
• Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers, Designers and Data Analyst to build scalable data driven products.
• Be responsible for developing backend APIs & working on databases to support the applications.
• Work in an Agile Environment that practices Continuous Integration and Delivery.
• Work closely with fellow developers through pair programming and code review process.

Skills

SQL

Pandas

R

ETL pipelines

SSIS

Python

AWS

GCP

REST API

Web scraping

Big data

Spark

Kafka

Hadoop

MongoDB

PostgreSQL

MySQL

Selenium

BeautifulSoup

CasperJS

PhantomJS

Node.js

Tools

BeautifulSoup

Selenium

CasperJS

PhantomJS

Node.js

AWS Lambda

ECS

Sparks

Spring

PostgreSQL

MongoDB

MySQL

Job description

Visa Hunt is seeking a data engineer to design, develop and deploy data tables, views and data marts across warehouses and data lakes. You will implement end-to-end data flows and contribute to data virtualization initiatives.

You will build scalable batch and real-time pipelines using SQL, Python and AWS services, ensuring data quality, governance and reliable integration across platforms. Collaborating with product, analytics and engineering teams, you will own APIs, data models and CI/CD