Senior Data Engineer

Innovatiq Engineering; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

Innovatiq Engineering is seeking a skilled Data Engineer to design, implement, and optimize data pipelines and reporting solutions. You will leverage Python, PySpark, SQL, and ETL tools to meet analytical needs and deploy BI dashboards using Tableau and SAP BO.

You will collaborate with cross‑functional teams to ensure reliable deployments, maintain data quality, and automate development and testing workflows using AI tools and DevOps practices.

Qualifications

• Good understanding and completion of projects using waterfall/Agile methodology.
• Working experience using Cloudera AI (Machine Learning), Plotly and data engineering AI tool stack
• Strong SQL, data modelling and data analysis skills are a must.
• Hands-on experience in big data engineering jobs using Python, Pyspark, Linux, and ETL tools like Informatica.
• Hands-on experience in a reporting or visualization tool like SAP BO and Tableau is must.
• Hands-on experience in DevOps deployment and data virtualisation tools like Denodo will be an advantage.
• Hand-on experience in scripting like python and shell scripts will be an advantage
• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.
• Good understanding of analytics and data warehouse implementations.
• Ability to troubleshoot complex issues ranging from system resource to application stack traces.
• Track record in implementing systems with high availability, high performance, high security hosted at various data centres or hybrid cloud environments will be an added advantage.
• Passion for automation, standardization, and best practices.
• AI and automation scope past experience will be an advantage.

Responsibilities

• Analyse the Authority’s data needs and document the requirements.
• Refine data collection/consumption by migrating data collection to more efficient channels.
• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.
• Develop test plan and scripts for system testing, support user acceptance testing.
• Build reports and dashboards according to user requirements
• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.
• Ensure the smooth operations and service level of IT solutions.
• Support production issues
• Apply AI coding tools such as Copilot and Coder AI to support development, testing, and code reviews.
• Use automation to improve development efficiency. This includes build, test, and deployment activities.
• Leverage AI assistance to improve code quality. This covers refactoring, defect resolution, and documentation.
• Demonstrate willingness to adopt new AI and automation tools where relevant to work outcomes.

Skills

SQL

Data modelling

Data analysis

Python

PySpark

Linux

ETL tools

Informatica

SAP BO

Tableau

Denodo

Hive

Impala

Cloudera Data Platform

DevOps

Tools

Plotly

Cloudera AI

Tableau

SAP BO

Informatica

Denodo

Job description

Key Responsibilities

• Analyse the Authority’s data needs and document the requirements.


• Refine data collection/consumption by migrating data collection to more efficient channels.


• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.


• Develop test plan and scripts for system testing, support user acceptance testing.


• Build reports and dashboards according to user requirements


• Work with the Authority’s technical teams to ensure smooth deployment and adoption of new solution.


• Ensure the smooth operations and service level of IT solutions.


• Support production issues


• Apply AI coding tools such as Copilot and Coder AI to support development, testing, and code reviews.


• Use automation to improve development efficiency. This includes build, test, and deployment activities.


• Leverage AI assistance to improve code quality. This covers refactoring, defect resolution, and documentation.


• Demonstrate willingness to adopt new AI and automation tools where relevant to work outcomes.

What we are looking for

• Good understanding and completion of projects using waterfall/Agile methodology.


• Working experience using Cloudera AI (Machine Learning), Plotly and data engineering AI tool stack


• Strong SQL, data modelling and data analysis skills are a must.


• Hands-on experience in big data engineering jobs using Python, Pyspark, Linux, and ETL tools like Informatica.


• Hands-on experience in a reporting or visualization tool like SAP BO and Tableau is must.


• Hands-on experience in DevOps deployment and data virtualisation tools like Denodo will be an advantage.


• Hand-on experience in scripting like python and shell scripts will be an advantage


• Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred.


• Good understanding of analytics and data warehouse implementations.


• Ability to troubleshoot complex issues ranging from system resource to application stack traces.


• Track record in implementing systems with high availability, high performance, high security hosted at various data centres or hybrid cloud environments will be an added advantage.


• Passion for automation, standardization, and best practices.


• AI and automation scope past experience will be an advantage.