Data Engineer: Real-Time Analytics & Pipelines

USER EXPERIENCE RESEARCHERS PTE. LTD; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

USER EXPERIENCE RESEARCHERS PTE. LTD in Singapore seeks a data-focused engineer to design, build, and deploy data tables, views, and pipelines across data warehouses, data lakes, and data virtualization.

You will perform data extraction, cleaning, transformation, and data flow management, with web scraping potentially part of the scope. Collaboration with PMs, architects, and analysts will be essential to deliver scalable, data-driven products.

Qualifications

• Proficient in data cleaning and transformation to ensure data accuracy.
• Proficient in building ETL pipelines with SSIS, AWS DMS, Python, and AWS services.
• Proficient in database design across SQL, PostgreSQL, MongoDB, and MySQL.
• Experience with cloud platforms such as GCP, AWS, and Azure.
• Knowledge of data warehousing, data lakes, and data virtualization.
• Familiar with REST APIs, web protocols, and big data tools.
• Familiar with web scraping tools like BeautifulSoup, Selenium.
• Comfortable with Windows and Linux development environments.
• Interest in bridging engineering and analytics.

Responsibilities

• Design, develop, and deploy data tables, views, and marts in data warehouses, data lakes, and data virtualization.
• Perform data extraction, cleaning, transformation, and data flow management; web scraping may be part of data extraction.
• Design, build, launch, and maintain large-scale batch and real-time data pipelines.
• Integrate and collate data silos in a scalable, compliant manner.
• Collaborate with PMs, architects, BAs, frontend devs, designers, and analysts.
• Develop backend APIs and work on databases to support applications.
• Work in an Agile environment with CI/CD practices.
• Pair program and participate in code reviews with fellow developers.

Skills

Data cleaning

ETL pipelines

SQL

Python

Cloud platforms

REST APIs

Big data tools

Scripting

Windows & Linux

Collaboration/Agile

Tools

SSIS

AWS DMS

AWS Lambda

ECS

EventBridge

AWS Glue

Spring

S3

PostgreSQL

MongoDB

Job description

USER EXPERIENCE RESEARCHERS PTE. LTD in Singapore seeks a data-focused engineer to design, build, and deploy data tables, views, and pipelines across data warehouses, data lakes, and data virtualization.

You will perform data extraction, cleaning, transformation, and data flow management, with web scraping potentially part of the scope. Collaboration with PMs, architects, and analysts will be essential to deliver scalable, data-driven products.