Data Engineer - G0 Fusion

Indigitise Pty Ltd; Canberra; On-site; AUD 120,000 - 180,000; Full time

Job summary

Indigitise Pty Ltd in Australia is seeking a skilled Senior Data Engineer to lead design and implementation of data integration pipelines for logistics and sustainment data.

You will collaborate with Business Analysts, Data Architects and developers to map data structures, migrate legacy warehouses to a modern data mesh, and develop domain-aligned data products for self-service analytics and governance across enterprise systems.

Qualifications

• Minimum 5 years dedicated data engineering experience in data warehouse contexts.
• SQL development including 3NF design and STAR schema implementation in data warehouse contexts.
• Experience with Agile development (Scrum).
• Python scripting for data engineering or data analysis.
• ETL/ELT pipeline execution and data quality controls.
• Datamart design, build and testing.
• Data Vault 2.0 patterns awareness (Desirable).

Responsibilities

• Design and implement data integration pipelines sourcing logistics and sustainment data from legacy applications.
• Collaborate with Business Analysts, Data Architects and developers to analyze data structures and develop data mappings.
• Migrate legacy data warehouse environments toward a Defence enterprise data mesh and develop domain-aligned data products for self-service analytics.
• Support data governance, security, privacy and lifecycle management across enterprise analytics.
• Develop conformed dimensions and reusable data products to support enterprise analytics requirements.

Skills

SQL development

STAR schema design

Agile/Scrum

Python scripting

ETL/ELT pipelines

Data quality controls

Datamart design

Data Vault 2.0

Tools

Cloudera data suite

Databricks

Job description

Under direction of the Commonwealth, undertake the following duties:

• Design and implement data integration pipelines that source logistics and sustainment data from legacy applications
• Collaborate with stakeholders including Business Analysts, Data Architects and developers, analyse logistics, maintenance, inventory, supply chain, procurement and asset management data structures and develop data mappings
• Generate and revise suggested data object designs across relational and data mesh environments as part of the prescribed data generation life cycle for Army analytical products
• Research and develop data migration strategies, cutover approaches and ongoing integration mechanisms between legacy data source and target data sources plus platform migration.
• Support testing, verification and operational acceptance of migrated logistics datasets.

Legacy Data Warehouse to Enterprise Data Mesh Migration

• Support the migration of analytical data assets, from Army legacy data warehouse into a Defence enterprise data mesh architecture.
• Design and develop domain-aligned data products that support self-service analytics, reporting and decision support capabilities.
• Transform landed data into centrally managed warehouse STAR schema plus governed, reusable data products aligned to Data Mesh principles.
• Support implementation of federated governance, data discoverability and enterprise-wide data sharing capabilities.

Data Product and Data Mart Development

• Design and develop business-ready data marts and analytical models supporting:
• Organisational structure and hierarchy management
• Logistics and inventory management
• Maintenance and sustainment management
• Equipment readiness and asset visibility
• Workforce and establishment reporting
• Operational planning and decision support
• Develop conformed dimensions and shared business entities to ensure consistency across analytical domains.
• Support data engineering for dimensional, data vault and hybrid modelling approaches as appropriate to support enterprise analytics requirements.
• Analyse enterprise semantic-ready datasets suitability for Army use cases.

Data Engineering and Platform Development

• Apply DevSecOps, CI/CD and pipeline execution where appropriate.
• Ensure compliance with enterprise governance, security, privacy and information management policies. Including standards for data quality, stewardship and lifecycle management, and master data management.

Requirements

• Minimum 5 years dedicated data engineering experience across:
• SQL development including 3NF design, STAR schema design and implementation, in a data warehouse context.
• Agile development, preferably Scrum.
• Experience migrating legacy data warehouse environments to preferably modern analytical platforms or experience with large application data migration or refactoring.
• Python scripting, data engineering or data analysis
• Data DevOps including ETL/ELT pipeline execution, data quality controls / verification
• Datamart design, build and testing.
• SQL code optimisation and refactoring.
• Python scripting and data transformation.

Desirable

• Experience with Data Vault 2.0 patterns, data mesh structures
• Experience with the logistics systems and logistics business domain, especially in a SAP enterprise context.
• Experience with Defence or related industries such as capability development and delivery, sustainment or operational planning.
• Experience with data engineering of sensitive datasets at SECRET, protected identities (PI) and personality identifiable information (PII)
• Working in a large team on a complex application in a decision support or data analytics context.
• Cloudera data analytics suite (CDV, CML etc) including Hive, Impala, Ozone, Iceberg, Spark or contemporary suite eg Databricks including data mesh concepts.