Senior Data Engineer

Data Knowhow; Canberra; Hybrid; AUD 120,000 - 160,000; Full time

Benefits offered by this job

12% superannuation

Job summary

Data Knowhow is seeking a Data Engineer in Canberra to design, develop, and optimise ETL pipelines using Microsoft technologies. You will work with multi-disciplinary teams to deliver data-rich products supporting analytics and BI capabilities within Azure.

The role requires strong data modelling, ETL lifecycle knowledge, excellent communication, and the ability to collaborate across stakeholders. Hybrid working arrangements are available for this contract position.

Qualifications

• Experience designing and orchestrating scalable ETL/ELT pipelines in Azure Data Factory.
• Strong T-SQL proficiency including stored procedures, functions and indexing.
• Experience with REST/SOAP APIs, OAuth2 and JSON parsing.
• Ability to collaborate across technical and business teams.
• Understanding of the Azure data ecosystem and security best practices.

Responsibilities

• Design and build ETL/ELT pipelines in Azure Data Factory and manage linked services, datasets, and integration runtimes.
• Write and optimize complex T-SQL and performance tuning; design dimensional models.
• Manage source control with Git and maintain Azure DevOps pipelines for deployment across environments.
• Monitor pipelines, set up alerts, and troubleshoot failures with logs and dashboards.
• Collaborate with data architects, modellers, analysts, and stakeholders to translate data needs into solutions.

Skills

ETL design

Analytical thinking

Stakeholder collaboration

Communications

SAFe / Agile

Education

Bachelor's or Master's in CS/Engineering/Data Science

Tools

Azure Data Factory

Azure DevOps

Git

Azure Monitor

Azure SQL / ADLS Gen2

SSIS/T-SQL

Job description

Job Description

Our Canberra-based federal government client is seeking to engage a suitably qualified and experienced Data Engineer to deliver and support a range of ICT projects/programs.

The Data Engineer will be required to work within multi-disciplinary project teams to design, develop, and optimise ETL and data pipelines producing analytical datasets embedded as part of data rich products supporting election systems. This role is pivotal in shaping the data architecture that underpins the client's analytics, forecasting and business intelligence capabilities leveraging the full power of the Microsoft ecosystem.

The ideal candidate will be an experienced Data Engineer with expertise in implementing analytical solutions using Microsoft technologies working closely with stakeholders and data modellers to develop an end-to-end solution.

They will be a self-starter with strong communication skills, a robust understanding of the data modelling and ETL lifecycle and have a bias towards collaborative problem solving.

The candidate must hold an active NV1 security clearance prior to commencement.

Estimated start date

Monday, 05 October 2026

Initial contract duration

05 October 2026 to 30 June 2027

Extension term

12 months

Number of extensions

1

Location of work

ACT, NSW

Working arrangements

Hybrid

Maximum hours

40 hours per week

Security clearance

Must be able to obtain Negative Vetting Level 1

Closing date

Monday August 31

Key duties and responsibilities

The Data Engineer will be responsible for, but not limited to:

• Design and build ETL/ELT pipelines in Azure Data Factory and create/manage linked services, datasets, and integration runtimes.
• Write and optimize complex T-SQL (stored procedures, functions, views) and performance tuning, query optimization, execution plans, indexing strategy
• Manage source control with Git (branching strategy, pull requests) and maintain Azure DevOps pipelines for automated deployment of ADF resources and infrastructure across Dev . Test . Prod environments.
• Monitor pipeline runs, set up alerts (Azure Monitor, Log Analytics) and troubleshoot failures and analyse logs, metrics, and monitoring dashboards to detect anomalies and failures early.
• Collaborate with data architects, data modellers, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Following the Scaled Agile Framework (SAFe) methodology for software development and contributing to Agile ceremonies.
• Production Support: Monitor, maintain, and support data integration workflows, pipelines, and ETL processes using Azure Data Factory.
• Incident Management: Respond to and resolve incidents, service requests, and system alerts related to data platform operations in a timely manner.
• Problem Resolution: Investigate, troubleshoot, and remediate data and infrastructure issues, working closely with development and infrastructure teams as needed.
• Ability to participate in on-call support rotations and respond to high-priority incidents as needed.
• Excellent analytical, problem-solving, and communication skills
• Adhering to coding best practices and the client's code quality standards including version control, code documentation, and testing.

Criteria

The client has specified that each candidate must provide a one page pitch to address all criteria specified. This is equal to 5000 characters.

• Demonstrated experience designing, developing, and orchestrating scalable ETL/ELT pipelines using Azure Data Factory — including parameterized, metadata-driven pipelines, integration runtimes, incremental loads, and robust error handling to move and transform data from diverse sources.
• Advanced ability to write, optimize, and troubleshoot complex T-SQL (stored procedures, functions, views), design dimensional data models (star/snowflake schemas), and perform performance tuning through indexing and query optimization.
• Practical experience ingesting and exchanging data via REST/SOAP APIs handling authentication (OAuth2, tokens, managed identities), pagination, rate limiting, and parsing JSON/XML responses into structured, analytics-ready datasets.
• Exceptional communication and documentation abilities, with a proven track record of collaborating across technical and business teams.
• Solid understanding of the wider Azure ecosystem (ADLS Gen2, Azure SQL, Key Vault, Azure Functions, Synapse) combined with strong analytical and troubleshooting skills to monitor pipelines, diagnose failures, and apply security best practices.
• Background in building and managing CI/CD pipelines using Azure DevOps, enabling streamlined deployment and operational efficiency.
• Prior experience in federal government data domains.
• Bachelor's or Master's degree in a relevant field (e.g. Computer Science, Data Science, Statistics, Engineering).

Why Data Knowhow?

As a contractor with Data Knowhow, you will be provided with mentoring and training to advance your career opportunities - we are keen to support you in your career journey and expand future opportunities for you. Data Knowhow provide expertise in data management, analytics and communication, with our staff mentoring and supporting you during your placement with our clients.

12% superannuation is provided on top of your requested daily rate if you are successful in this position.

We are continually expanding our offerings to make life as a Data Knowhow employee a valuable experience.