Senior Data Engineer

Data Knowhow; Canberra; Hybrid; AUD 110,000 - 140,000; Full time

Benefits offered by this job

Mentoring and training

12% superannuation

Job summary

Data Knowhow is seeking a seasoned Data Engineer in Canberra to design, build, and support ETL/ELT pipelines using Azure Data Factory and SQL-based data models. You will collaborate with data architects, modellers, and business stakeholders to deliver analytics-ready datasets for analytics and BI solutions.

The role requires strong communication, a solid grasp of the data modelling lifecycle, and experience with Azure services such as ADLS Gen2, Azure SQL, Key Vault, Functions, and Synapse.

Qualifications

• Experience designing, developing, and orchestrating scalable ETL/ELT pipelines in Azure Data Factory.
• Advanced ability to write and troubleshoot complex T‑SQL (procedures, functions, views) and optimize queries.
• Experience exchanging data via REST/SOAP APIs with authentication and parsing JSON/XML responses.
• Strong communication and documentation skills across technical and business teams.

Responsibilities

• Design and build ETL/ELT pipelines in Azure Data Factory with linked services, datasets, and integration runtimes.
• Write, optimize, and troubleshoot T-SQL with indexing strategy and performance tuning.
• Maintain source control with Git and manage Azure DevOps pipelines for automated deployments.
• Monitor pipelines, set up alerts, troubleshoot failures, and analyze logs and metrics.
• Collaborate with data architects, modellers, analysts, and stakeholders to translate requirements into solutions.
• Contribute to SAFe/agile ceremonies and participate in on-call support as needed.
• Provide production support and incident resolution for data platform operations.

Skills

ETL/ELT pipelines

T-SQL performance tuning

analytical problem solving

stakeholder collaboration

communication skills

Education

Bachelor's degree in Computer Science

Master's degree in a relevant field

Tools

Azure Data Factory

Git

Azure DevOps

ADLS Gen2

Azure SQL

Key Vault

Azure Functions

Azure Synapse

Job description

Our Canberra-based federal government client is seeking to engage a suitably qualified and experienced Data Engineer to deliver and support a range of ICT projects/programs.

The Data Engineer will be required to work within multi-disciplinary project teams to design, develop, and optimise ETL and data pipelines producing analytical datasets embedded as part of data rich products supporting election systems. This role is pivotal in shaping the data architecture that underpins the client’s analytics, forecasting and business intelligence capabilities leveraging the full power of the Microsoft ecosystem.

The ideal candidate will be an experienced Data Engineer with expertise in implementing analytical solutions using Microsoft technologies working closely with stakeholders and data modellers to develop an end-to-end solution.

They will be a self-starter with strong communication skills, a robust understanding of the data modelling and ETL lifecycle and have a bias towards collaborative problem solving.

The candidate must hold an active NV1 security clearance prior to commencement.

Estimated start date

Monday, 05 October 2026

Initial contract duration

05 October 2026 to 30 June 2027

Extension term

12 months

Number of extensions

1

Location of work

ACT, NSW

Working arrangements

Hybrid

Maximum hours

40 hours per week

Security clearance

Must be able to obtain Negative Vetting Level 1

Closing date

Monday August 31

Key duties and responsibilities

The Data Engineer will be responsible for, but not limited to:

• Design and build ETL/ELT pipelines in Azure Data Factory and create/manage linked services, datasets, and integration runtimes.
• Write and optimize complex T-SQL (stored procedures, functions, views) and performance tuning, query optimization, execution plans, indexing strategy
• Manage source control with Git (branching strategy, pull requests) and maintain Azure DevOps pipelines for automated deployment of ADF resources and infrastructure across Dev . Test . Prod environments.
• Monitor pipeline runs, set up alerts (Azure Monitor, Log Analytics) and troubleshoot failures and analyse logs, metrics, and monitoring dashboards to detect anomalies and failures early.
• Collaborate with data architects, data modellers, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
• Following the Scaled Agile Framework (SAFe) methodology for software development and contributing to Agile ceremonies.
• Production Support: Monitor, maintain, and support data integration workflows, pipelines, and ETL processes using Azure Data Factory.
• Incident Management: Respond to and resolve incidents, service requests, and system alerts related to data platform operations in a timely manner.
• Problem Resolution: Investigate, troubleshoot, and remediate data and infrastructure issues, working closely with development and infrastructure teams as needed.
• Ability to participate in on-call support rotations and respond to high-priority incidents as needed.
• Excellent analytical, problem-solving, and communication skills
• Adhering to coding best practices and the client’s code quality standards including version control, code documentation, and testing.

Criteria

The client has specified that each candidate must provide a one page pitch to address all criteria specified. This is equal to 5000 characters.

• Demonstrated experience designing, developing, and orchestrating scalable ETL/ELT pipelines using Azure Data Factory — including parameterized, metadata-driven pipelines, integration runtimes, incremental loads, and robust error handling to move and transform data from diverse sources.
• Advanced ability to write, optimize, and troubleshoot complex T-SQL (stored procedures, functions, views), design dimensional data models (star/snowflake schemas), and perform performance tuning through indexing and query optimization.
• Practical experience ingesting and exchanging data via REST/SOAP APIs handling authentication (OAuth2, tokens, managed identities), pagination, rate limiting, and parsing JSON/XML responses into structured, analytics-ready datasets.
• Exceptional communication and documentation abilities, with a proven track record of collaborating across technical and business teams.
• Solid understanding of the wider Azure ecosystem (ADLS Gen2, Azure SQL, Key Vault, Azure Functions, Synapse) combined with strong analytical and troubleshooting skills to monitor pipelines, diagnose failures, and apply security best practices.
• Background in building and managing CI/CD pipelines using Azure DevOps, enabling streamlined deployment and operational efficiency.
• Prior experience in federal government data domains.
• Bachelor's or Master's degree in a relevant field (e.g. Computer Science, Data Science, Statistics, Engineering).

Why Data Knowhow?

As a contractor with Data Knowhow, you will be provided with mentoring and training to advance your career opportunities - we are keen to support you in your career journey and expand future opportunities for you. Data Knowhow provide expertise in data management, analytics and communication, with our staff mentoring and supporting you during your placement with our clients.

12% superannuation is provided on top of your requested daily rate if you are successful in this position.

We are continually expanding our offerings to make life as a Data Knowhow employee a valuable experience.