Reporting and Analytics Developer/Data Engineer 0642B

User Experience Researchers Pte.Ltd; Singapore; On-site; SGD 80,000 - 120,000; Full time

Job summary

A leading company in user experience research is seeking a Big Data Engineer to design and maintain data processing jobs. The ideal candidate will possess strong SQL, Python, and big data engineering skills, contributing to effective data solutions. This role requires a passion for automation and best practices in a dynamic team environment.

Qualifications

• Minimum 4 years of experience in big data environments.
• Hands-on experience in big data engineering with Python, Pyspark.
• Experience with reporting/visualization tools and DevOps deployment.

Responsibilities

• Design, develop, and maintain data ingestion and processing jobs.
• Analyze data needs and document requirements.
• Build reports and dashboards according to user requirements.

Skills

SQL

Data modeling

Data analysis

Python

Pyspark

ETL

Agile methodology

DevOps

Data visualization

Tools

Informatica

SAP BO

Tableau

Hive

Impala

Cloudera Data Platform

Denodo

Job description

We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have a minimum of 4 years of experience managing data engineering jobs in big data environments, e.g., Cloudera Data Platform. The successful candidate will be responsible for designing, developing, and maintaining data ingestion and processing jobs. They will also be integrating data sets to provide seamless data access to users.

Responsibilities

• Analyze the Authority's data needs and document the requirements.
• Refine data collection/consumption by migrating data collection to more efficient channels.
• Plan, design, and implement data engineering jobs and reporting solutions to meet analytical needs.
• Develop test plans and scripts for system testing, support user acceptance testing.
• Build reports and dashboards according to user requirements.
• Work with the Authority's technical teams to ensure smooth deployment and adoption of new solutions.
• Ensure smooth operations and service levels of IT solutions.
• Support production issues.

What we are looking for:

• Good understanding and experience with projects using waterfall/Agile methodology.
• Strong SQL, data modeling, and data analysis skills.
• Hands-on experience in big data engineering using Python, Pyspark, Linux, and ETL tools like Informatica.
• Experience with reporting or visualization tools like SAP BO and Tableau.
• Experience with DevOps deployment and data virtualization tools like Denodo is a plus.
• Experience with Hive, Impala, and Cloudera Data Platform is preferred.
• Good understanding of analytics and data warehouse implementations.
• Ability to troubleshoot complex issues from system resources to application stack traces.
• Experience implementing systems with high availability, performance, and security in data centers or hybrid cloud environments is advantageous.
• Passion for automation, standardization, and best practices.