Senior Data Engineer: AI-Driven Data & Analytics

INNOVATIQ ENGINEERING PTE. LTD.; Singapore; On-site; SGD 70,000 - 120,000; Full time

Job summary

INNOVATIQ ENGINEERING PTE. LTD. is seeking a data engineering professional to analyze data needs and design end-to-end data pipelines and dashboards. You will migrate data collection to efficient channels and implement scalable reporting solutions.

You will use Python, PySpark and SQL to build ETL processes, work with Cloudera/Data Platform, and deploy solutions with DevOps practices. Familiarity with AI tools like Copilot and Coder AI is beneficial.

Qualifications

• Experience delivering projects using waterfall or Agile methodologies.
• Strong SQL, data modeling and data analysis skills are required.
• Hands-on experience in big data engineering with Python, PySpark, Linux and ETL tools like Informatica.
• Hands-on experience in reporting/visualization tools like SAP BO and Tableau.
• DevOps deployment and data virtualization tools like Denodo are a plus.
• Scripting in Python and shell scripts is advantageous.
• Experience with Hive, Impala and Cloudera Data Platform preferred.
• Understanding of analytics and data warehouse implementations.
• Ability to troubleshoot complex issues from system resources to application traces.
• Experience with high availability, performance, and security across data centers or hybrid clouds.
• Passion for automation, standardization and best practices.
• Experience with AI and automation tools is advantageous.

Responsibilities

• Analyse data needs and document requirements.
• Refine data collection by migrating to efficient channels.
• Plan, design and implement data engineering jobs and reporting solutions.
• Develop test plans and scripts for system testing and UAT.
• Build reports and dashboards per user requirements.
• Collaborate with IT teams for deployment and adoption of new solutions.
• Ensure smooth operations and service levels of IT solutions.
• Support production issues.
• Apply AI coding tools to support development, testing and reviews.
• Use automation to improve build, test and deployment.
• Leverage AI to improve code quality, refactoring, defect resolution and documentation.
• Demonstrate willingness to adopt new AI/automation tools where relevant.

Skills

SQL

Data modeling

Data analysis

Python

PySpark

Linux

ETL tools

Tableau

Cloudera Data Platform

Hive/Impala

Plotly

AI coding tools

Tools

Informatica

SAP BO

Tableau

Denodo

Hive

Impala

Cloudera Data Platform

Job description

INNOVATIQ ENGINEERING PTE. LTD. is seeking a data engineering professional to analyze data needs and design end-to-end data pipelines and dashboards. You will migrate data collection to efficient channels and implement scalable reporting solutions.

You will use Python, PySpark and SQL to build ETL processes, work with Cloudera/Data Platform, and deploy solutions with DevOps practices. Familiarity with AI tools like Copilot and Coder AI is beneficial.