Data Engineer (Data & Systems Engineering)

re-zoo-me; Singapore; On-site; SGD 60,000 - 100,000; Full time

Job summary

CSIT in Singapore is seeking a Data Engineer to join the Data & Systems Engineering team. You will develop and maintain data pipelines, optimize data delivery, and design scalable infrastructure while collaborating with cross-functional teams to gather requirements and build cutting-edge data products.

The ideal candidate has 2+ years of hands-on experience with Python, SQL/NoSQL, and ETL tools like NiFi or Airflow, and familiarity with microservices and RESTful API design.

Qualifications

• Background in Computer Science, Computer Engineering, Information Systems, or a related field.

Responsibilities

• Develop and maintain robust data pipelines for extracting, transforming, and loading data from diverse sources.
• Identify, design, and implement process improvements, including automation of manual processes, optimization of data delivery, and design of scalable infrastructure.
• Gather and analyze telemetry data to provide actionable insights, enhancing the efficiency of data systems and key business performance metrics.

Skills

Python

Golang

Java

C++

SQL

NoSQL

REST API design

Microservices

Agile SCRUM

Data pipelines

Education

Bachelor's degree in Computer Science / Computer Engineering / Information Systems or related field

Tools

NiFi

Airflow

Elasticsearch

Solr

Hadoop

Spark

Kafka

Job description

We are seeking a Data Engineer to join our Data & Systems Engineering team. In this role, you will be responsible for enhancing and optimizing the data flow between our systems and refining our data pipeline architecture. You will collaborate closely with cross-functional teams to gather requirements, design scalable solutions, and implement cutting-edge data products. The ideal candidate is passionate about driving the development of our next generation of data initiatives.

Key Responsibilities

• Develop and maintain robust data pipelines for extracting, transforming, and loading data from diverse sources.
• Identify, design, and implement process improvements, including automation of manual processes, optimization of data delivery, and design of scalable infrastructure.
• Gather and analyze telemetry data to provide actionable insights, enhancing the efficiency of data systems and key business performance metrics.

Minimum Requirements

• Background in Computer Science, Computer Engineering, Information Systems, or a related field.
• A minimum of 2 years of hands-on experience with object-oriented programming languages such as Python, Golang, Java, or C++. (Project/internship experience will be considered for recent graduates.)
• Proficiency with both relational SQL and NoSQL databases.
• Experience in building ETL flows using tools such as NiFi or Airflow.
• Familiarity with microservices architecture and RESTful API design.

Preferred Qualifications

• Experience with software development in Agile SCRUM environments, including testing and production error/metrics monitoring.
• Expertise in building scalable, fault-tolerant, and maintainable data pipelines.
• Proficiency in big data tools such as Hadoop, Spark, and Kafka.
• Experience with search engines like Elasticsearch or Solr.

Considerations for senior positions

• Candidates with at least 5 years of proven track record in data engineering development.
• Experience in technical coaching to junior data engineers.

As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans.