Senior Data Engineer & Pipeline Architect

PERSOL SINGAPORE PTE. LTD.; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Persol Singapore Pte. Ltd. is seeking a Data Engineer to design and implement data pipelines, optimize SQL queries, and collaborate with analysts and scientists to deliver reliable data solutions.

You will organize data stores on Cloud platforms, build scalable pipelines for large volumes, and develop high-performance architectures while coordinating with business stakeholders to gather and translate requirements into concrete implementations.

Qualifications

• Bachelor’s degree in CS/IT or related field.
• 5 years of data engineering experience.
• Strong SQL, ETL Teradata, Python and data extraction/manipulation skills.
• Experience with Kafka messaging.
• Experience in Informatica development.
• Experience implementing complex ETL and data extraction projects from scratch.
• Fluency in Python, Spark, HiveQL, Hadoop ecosystem, and data-centric workflow tools; Unix/Linux/HDFS knowledge.
• Proficiency in SQL and RDBMS.
• Good working knowledge of Git and CI/CD pipelines.
• Experience with Graph databases is desirable but not required.
• Thorough knowledge of data security, privacy, and compliance regulations.

Responsibilities

• Assist design and implementation of data pipelines to collect, process, and store data from various sources.
• Write and optimize SQL queries to extract and manipulate data for reporting and analysis.
• Collaborate with data analysts and data scientists to understand data requirements and provide data support.
• Organize, build and maintain data stores of structured and unstructured data on a Cloud environment.
• Build data pipelines to enable scalable processing of large data volumes.
• Develop scalable, high-performance application architectures using modern frameworks.
• Work with stakeholders to gather requirements and convert them into solutions; monitor data pipelines for quality and reliability.
• Enhance and maintain existing applications, identify optimization opportunities.
• Document data processes, workflows, and models; maintain deployment-related documentation.

Skills

SQL

ETL

Python

Kafka

Git

CI/CD

Spark

HiveQL

Hadoop

Unix/Linux

RDBMS

Graph DB

Education

Bachelor’s degree in Computer Science / IT or related field

Tools

Informatica

Kafka

Git

CI/CD pipelines

Hadoop ecosystem

Spark

Job description

Persol Singapore Pte. Ltd. is seeking a Data Engineer to design and implement data pipelines, optimize SQL queries, and collaborate with analysts and scientists to deliver reliable data solutions.

You will organize data stores on Cloud platforms, build scalable pipelines for large volumes, and develop high-performance architectures while coordinating with business stakeholders to gather and translate requirements into concrete implementations.