Senior Data Engineer

WAN SOLUTIONS; Singapore; On-site; SGD 120,000 - 200,000; Full time

Job summary

WAN SOLUTIONS seeks a Senior Data Engineer to design, implement and maintain scalable data pipelines and warehousing that support batch and real-time processing. You will collaborate with stakeholders, data scientists and partners to build robust data models and secure data flows.

The role focuses on data quality, governance and performance across diverse sources, demanding 3+ years in DI/ETL and hands-on SQL, Python, and data platform experience.

Qualifications

• 3+ years building and optimizing data pipelines using DI/ETL technologies such as Informatica or Talend.
• Experience with data transformation, metadata, dependency and workload management and big data tech (Spark, Hadoop, Hive).
• Proficiency with SQL, Python and multiple data sources (flat files, SQL, SAP/Postgres, unstructured data).

Responsibilities

• Collaborate with customers, partners and colleagues on data-related technical issues and support their data infrastructure needs.
• Work closely with data scientists to gather data requirements for modeling work.
• Design, develop, document, manage and maintain data models, ETL processes, data warehouse and data pipelines for large volumes of data from disparate sources and latencies.
• Define, monitor and report SLAs for data pipelines and data products.
• Understand data security and governance standards to implement compliant solutions.
• Lead data quality assurance practices.
• Support data management solutions pre-sales initiatives and post-sales support.

Skills

Data Wrangling

Data Visualization

DBMS

SQL

Python

Tools

Informatica

Talend

Spark

Hadoop

Hive

MS Access

SharePoint

Power BI

Kubernetes

Docker/Podman

Job description

We are seeking a Senior Data Engineer that supports the design, implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable and secure manner. The Senior Data Engineer focuses on defining optimal solutions to data collection, processing and warehousing. He / She designs, codes and tests data systems and works on implementing those into the internal infrastructure.

What You’ll Do

• Work with stakeholders including customers, partners and colleagues on data-related technical issues and support their data infrastructure needs;
• Work closely with data scientists to solicit data requirements to support modeling works
• Design, develop, document, manage and maintain data models, ETL processes, data warehouse, data management and pipeline solutions for large volume of structured/unstructured data from disparate sources and with different latencies (e.g. on-demand, batch, real-time, near-real-time);
• Define, monitor and report SLAs for data pipelines and data products;
• Understand data security and governance standards or requirements to implement solutions that ensure adherence to these standards or meet such requirements;
• Drive/execute data quality assurance practices; and
• Support data management solutions pre-sales initiatives, proposal development and provide post-sales support.

What You’ll Bring

• Shall have at least three (3) years’ experience in building and optimising data pipelines, architecture and datasets using DI/ETL technology (Informatica, Talend etc).
• Shall have experience in supporting data transformation, data structures, metadata, dependency and workload management, big data technologies (e.g., Sparks, Hadoop, Hive).
• Shall have the following skillsets: Data Wrangling, Data Visualization, DBMS, SQL, PythonExperience in multiple data source support (flat files, SQL database, SAP database, PostgreSQL database, unstructured data not limited to text, documents, images, digitalised video, digitalised audio, sensor data) would be advantageous.
• Experience in these products/tools would be advantageous: MS Access (Front-end), SharePoint (Back-end), Power BI (Visualisation), Kubernetes, Docker/Podman, Microservices Architecture, DevSecOps Methodology

EA License No: 12C6045 | UEN: 53041699M