Senior Big Data Engineer – Python, Spark & Cloudera

AvePoint; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

AvePoint is seeking a highly skilled Big Data Engineer in Singapore to design, develop, and maintain data ingestion and processing jobs for business requirements. You will integrate data sets to provide seamless access and support analytics across environments.

The ideal candidate has 4+ years in data engineering, hands-on experience with PySpark, Python, Hive/Impala, and CDP-like platforms, with strong SQL and data modelling skills to drive scalable solutions.

Qualifications

• 4+ years of experience in data engineering in big data environments.
• Experience with Cloudera Data Platform or similar.
• Strong SQL, data modelling and data analysis skills.
• Hands-on with Python, PySpark, Linux and ETL tools.

Responsibilities

• Analyze the Authority’s data needs and document the requirements.
• Refine data collection/consumption by migrating data collection to more efficient channels.
• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.
• Develop test plan and scripts for system testing, support user acceptance testing.
• Work with IT teams to ensure smooth deployment and adoption of new solution.
• Ensure the smooth operations and service level of IT solutions.
• Support production issues - Track record in implementing systems with high availability, high performance, high security.

Skills

SQL

Data modelling

Data analysis

Communication

Tools

Python

PySpark

Linux

Informatica

Hive

Impala

Denodo

Tableau

SAP BO

Job description

AvePoint is seeking a highly skilled Big Data Engineer in Singapore to design, develop, and maintain data ingestion and processing jobs for business requirements. You will integrate data sets to provide seamless access and support analytics across environments.

The ideal candidate has 4+ years in data engineering, hands-on experience with PySpark, Python, Hive/Impala, and CDP-like platforms, with strong SQL and data modelling skills to drive scalable solutions.