Data Engineer - Big Data

Synapxe; Queenstown; On-site; NZD 90,000 - 130,000; Full time

Job summary

A leading data solutions provider in New Zealand is seeking a Senior Data Strategist responsible for developing the data strategy, overseeing data preparation, and implementing data management standards. Candidates should have extensive experience in data management, cloud-based solutions, and effective collaboration with diverse stakeholders. This role involves working independently to ensure data quality and efficiency.

Qualifications

• Minimum eight years of experience in data management or advanced analytics solutions.
• Proficiency in ETL hardware/software products and methodologies.
• Experience in cloud platforms and data solutions.

Responsibilities

• Develop TRUST data strategy with various stakeholders.
• Oversee data preparation and provisioning in MCDR.
• Implement common data model and data quality programme for TRUST.

Skills

Data management

Cloud-based data solutions

ETL development

Structured and unstructured data analytics

Data quality tools

Education

Degree/master’s in computer science or related field

Tools

AWS

Informatica

SQL

Job description

Roles and responsibilities

1) Develop TRUST data strategy:

Work with stakeholders to understand data analytics needs, data structure requirements (both in terms of scalability and accessibility), and translate this into a coherent near to long term data strategy for TRUST.

Support translation of data business needs into technical system requirements for MCDR, in terms of collection, storage, batch and real-time processing, as well as analysis of information from structured and unstructured sources in a scalable, repeatable, and secure manner.

Identify opportunities for improvements and optimisation e.g., Implement best practices and performance optimization on Big Data and Cloud to achieve the best data engineering outcomes.

2) Oversee data preparation and data provisioning for TRUST:

Collaborate with data engineers to organise and prepare anonymised datasets in MCDR according to TRUST standards, and then providing the data in accordance with the approved TRUST Data Request. This involves working with the data engineers closely to ensure that the datasets meet the required standards and are made available as per the specific data request guidelines set by TRUST.

3) Oversee implementation of common data model and data quality programme in TRUST and MCDR:

Work with data analysts, data scientists, clinicians and other stakeholders to implement common data models to support analytics use cases.

Design and implement tools to enhance the data strategy and enable seamless integration with the data, potentially leveraging API calls for efficient integration.

Implements data management standards and practices.

Requirements

• Degree/master’s in computer science, Information Technology, Computer Engineering or equivalent.
• At least eight (8) years of relevant working experience in Data management / Integration / Modelling the data warehouse or advanced analytics solutions.
• Demonstrate good, in-depth knowledge in relevant Extract-Transform-Load (ETL) hardware/software products, frameworks, and methodologies.
• Experience in designing and implementing cloud-based data solutions using cloud platforms (e.g., AWS cloud native tools)
• Experience with at least two of the following areas:
  • Databases (e.g., Oracle, MS SQL, MySQL, Teradata)
• Big data (e.g., Hadoop ecosystem)
  • ETL development using ETL tools (e.g., Informatica,IBM DataStage, Talend)
  • Data repository design (e.g., operational data stores, dimensional data stores, data marts)
  • Data interrogation techniques (e.g., SQL, NoSQL).
  • Structured and unstructured data analytics.
  • Batch and real-time data ingestion and processing
  • Data quality tools and processes.
  • Data transformation and terminology equivalence mapping.
    • Experience in data modelling for analytics (e.g.,star schemas, snowflake schemas, OMOP CDM).
    • Experience in interacting with analytics stakeholders(economists, statisticians, clinicians, policy makers) on a business or domain level.
    • Comfortable working independently to carry out data analysis, estimate data quality and sufficiency.
    • Good interpersonal skills, a detail-oriented & flexible person who can work across different areas within the team.
    • The following will be preferred: Some understanding of Singapore Healthcare System and healthcare data governance,management;