Senior Data Engineer

SINGTEL MOBILE SINGAPORE PTE. LTD.; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading telecommunications company in Singapore is seeking a skilled Data Engineer to develop and enhance data pipelines on Big Data platforms. The role requires at least 5 years of experience in data solutions, proficiency in tools like Hadoop and Apache Airflow, and the ability to lead junior engineers. This is a fantastic opportunity to make a significant impact in a dynamic environment.

Qualifications

• 5 years of hands-on development experiences.
• Experience in Open Source tools for Data Analysis.
• Knowledge of metadata and data management.

Responsibilities

• Develop and improve data pipelines on Big Data platforms.
• Lead and monitor Junior Data Engineers.
• Design solutions aligned with data architecture principles.

Skills

Big Data Platform Knowledge

Python

Spark

Data API Exposure

Leadership

Education

Bachelor degree in Computer Science or related field

Tools

Hadoop

Apache Airflow

SQL

Java

Job description

Make an Impact by:

• Develop new and improve existing data pipeline on Big Data platform (Hadoop, mapR or equivalent).
• Build new and enhance existing data application for streaming and batch datasets.
• Work on Apache Airflow on data pipeline, Python, Spark, PySpark, Scala, Java, SQL, etc for data application open sources technologies.
• Responsible in developing frontend dashboards and integration to back end. Using frontend and backend API Frameworks such as React, Angular Django, FastAPI, Springboot.
• Perform R&D and conduct PoC (proof-of-concept) for new data solution.
• Perform metadata and data management, data security.
• Drive optimization, testing and tooling to improve data quality & efficiency in data lake and streaming platform.
• Lead and monitor the performance of Junior Data Engineer and providing them with practical guidance, solution validation and implementation.
• Lead and manage DevOps, DataOps and Streaming Operations (or equivalent) in Data Engineering.
• Design high level & detailed design to ensure that the solution delivers to the business needs and align to the data & analytics architecture principles and roadmap.
• Collaborate with different stakeholders from business, technical, project management and operation to design and implement the solution.
• Ability to lead troubleshooting efforts for complex design and eliminate application issue faced by the project and operation team.

Skills for Success:

• Bachelor degree in Computer Science, Math, Data Analytics or related field with at least 5 years of hands-on development experiences.
• Knowledge in Big Data Platform such as Hadoop, mapR, Cloudera, HPE, etc.
• Hands-on experience in Programming language including Python, PysSpark, SQL, noSQL, kSQL, prefer to have experience in Big Data application and Linux shell scripting.
• Experience in Data API exposure (Spring Boot, Flask or equivalent) and web development.
• Experience in Open Source like Apache Airflow, Zeppelin Notebook for Data Exploratory Analysis, etc