Lead Data Engineer (Snowflake, Dbt) – 12-Month Fixed-Term Contract
Commonwealth Bank Of Australia
Receba uma resposta desta empresa empregadora
Seu currículo e carta de apresentação personalizados exatamente para o que estão contratando.
Benefícios oferecidos por esta vaga
Resumo da vaga
Commonwealth Bank of Australia is seeking a Lead Data Engineer to drive design, build and operation of data pipelines for AML/CTF, sanctions and other financial crime initiatives. You will lead a small team of engineers and analysts, contribute to code, and set engineering standards across Snowflake, dbt and cloud/on‑prem environments.
You bring hands-on data engineering capability with strong SQL, Python and experience in Snowflake, dbt, and Databricks.
Qualificações
- Hands-on data engineering across production data pipelines.
- Strong SQL, Python and data modelling skills.
- Experience with Snowflake, dbt and data integration tools.
Responsabilidades
- Lead a small team of 3–4 data engineers, providing technical direction.
- Design, build and optimise production-grade data pipelines.
- Review code, establish engineering standards and enable delivery.
- Collaborate with stakeholders to translate financial crime requirements into data solutions.
- Ensure data quality, monitoring and secure data access.
Conhecimentos
Ferramentas
Descrição da vaga
Showing 1707 Lead Data Engineer Snowflake Dbt 12 Month Fixed Term Contract jobs in Sydney
You will be part of a specialist team reporting to the Executive Manager, Financial Crime Compliance, with a primary focus on supporting ASB with the delivery of data engineering capabilities for Transaction Monitoring and broader financial crime initiatives.
This is a hands-on Lead Data Engineer role . You will act as both a senior individual contributor and the technical and people lead for a small team of 34 data engineers.
You will be expected to design solutions, write and review production code, build and troubleshoot data pipelines, and contribute directly to engineering delivery , while also setting technical direction, establishing engineering standards and leading the day-to-day delivery of the team.
You will work closely with stakeholders across ASB, FCC Australia and New Zealand, and technology teams to translate complex financial crime and business requirements into reliable, scalable and well-controlled data solutions.
- Act as a senior hands-on engineer , designing, building, testing, deploying and supporting data engineering solutions.
- Lead and line-manage a small team of 34 data engineers, analysts and QAs providing technical direction, coaching, feedback and development.
- Lead delivery of ASB data engineering initiatives while also contributing directly to the codebase and technical implementation.
- Design, build and optimise production-grade data pipelines, transformations and integration solutions.
- Write high-quality, maintainable SQL and dbt code and contribute directly to complex engineering work where required.
- Review code produced by the team, provide constructive technical feedback and ensure agreed engineering standards are consistently applied.
- Lead technical and solution design reviews and make pragmatic engineering decisions where requirements, designs or delivery approaches are unclear.
- Establish and maintain engineering standards covering coding, testing, documentation, source control, deployment, monitoring and support.
- Translate business, data and financial crime requirements into practical end-to-end technical solutions.
- Work across Snowflake, dbt, IBM StreamSets and related technologies to support cloud, on-premises and Oracle-integrated data patterns.
- Diagnose and resolve complex data, performance and integration issues, including production problems where required.
- Build data quality, reconciliation, load assurance and control requirements into engineering solutions.
- Identify technical risks, dependencies and delivery constraints early and drive them through to resolution.
- Work closely with engineers, architects, database teams, business SMEs and technology stakeholders to support successful delivery.
- Promote reusable engineering patterns, automation and continuous improvement that improve delivery speed, maintainability and control.
- Ensure solutions are secure, resilient, scalable and aligned with relevant enterprise technology and engineering standards.
You are an experienced senior or lead data engineer who combines strong hands-on engineering capability with the ability to lead and drive delivery through a team.
You are comfortable moving between hands‑on technical work, code and design reviews, troubleshooting complex data problems, coaching engineers and engaging with senior business and technology stakeholders.
- Strong and recent hands-on data engineering experience , including personally building and supporting production-grade data pipelines.
- Advanced SQL skills and strong experience with cloud data platforms, ideally Snowflake .
- Strong hands‑on experience with dbt , including developing transformations, models, tests and reusable engineering components.
- Demonstrated experience writing production code and remaining technically hands‑on while leading other engineers.
- Experience providing technical leadership to data engineers, including code review, solution design and engineering standards.
- Strong understanding of software and data engineering practices including Git/source control, CI/CD, automated testing, deployment and production support.
- Experience implementing data quality, reconciliation, load assurance, monitoring and control mechanisms.
- Exposure to IBM StreamSets or similar data movement and integration technologies.
- Experience working across cloud and on-premises integration patterns; Oracle experience would be advantageous.
- Strong debugging and problem-solving skills, including the ability to investigate complex data and pipeline issues.
- Ability to work through ambiguous requirements, identify gaps, challenge proposed approaches and shape fit-for-purpose solutions.
- Strong stakeholder management skills and the confidence to provide technical direction across engineering, architecture, business and operational teams.
- Experience in financial services, Financial Crime or Transaction Monitoring would be advantageous but is not essential.
At CommBank, we're committed to creating an accessible, inclusive and respectful workplace. If you require support or adjustments, please let us know.
We welcome applications from people of all backgrounds, and we're particularly committed to making a positive difference for Aboriginal and/or Torres Strait Islander Peoples. For support, please contact .
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on .
For 20-plus years we have paired world-class engineering with deep industry insight, helping global brands create intelligent digital experiences that drive meaningful change. Join us as we expand our Australian Data & AI capability and help clients unlock the full value of their data assets.
We’re looking for a Lead Data Engineer to help our clients design, build and operate reliable data platforms that turn complex data into useful business insights.
You’ll work across the data lifecycle, from architecture and source integration through to data pipelines, semantic models, visualisation and reporting. You’ll bring strong engineering skills in Python and SQL, alongside hands‑on experience with at least one of Databricks, Snowflake or Microsoft Fabric.
Working closely with clients, analysts, data scientists and fellow engineers, you’ll translate business requirements into secure, maintainable and scalable data solutions. You’ll also provide technical guidance, contribute to design decisions and help the team strengthen its engineering practices.
- Work directly with clients to translate business requirments into technical solutions; Providing guidance and contributing to design decisions.
- Design, build and maintain data pipelines to ingest, transform and deliver data from a range of sources, including APIs.
- Develop solutions using Databricks, Snowflake or Microsoft Fabric, selecting approaches that suit the client’s requirements and existing environment.
- Contribute to data architecture and solution design, including data storage, integration and modelling.
- Develop data models, ontologies and semantic layers that establish consistent business definitions and support analytics and reporting.
- Implement pipeline orchestration, scheduling, monitoring and operational support to keep data services reliable.
- Troubleshoot data and pipeline issues, improving performance, scalability and cost efficiency.
- Apply data quality controls, automated testing and validation throughout the data lifecycle.
- Implement role-based access control and follow security best practices for data platforms and integrations.
- Apply DevOps practices to data engineering, including version control, repository management, CI/CD and automated deployment.
- Build data visualisations and reports that make information clear and useful to business users.
- Document solutions, review code and share knowledge to support maintainable delivery and continuous improvement.
- Hands‑on experience delivering production data solutions using one or more of Databricks, Snowflake or Microsoft Fabric.
- Strong programming skills in Python and SQL.
- Experience designing, developing and operating data pipelines, including ETL/ELT processes, orchestration and monitoring.
- A sound understanding of data architecture and modelling techniques, including analytical and dimensional modelling.
- An understanding of ontologies and semantic modelling, and how they connect data structures to business concepts.
- Practical experience optimising data processing and troubleshooting pipeline, query and platform performance.
- Experience integrating data through APIs, including authentication, error handling and reliable data exchange.
- Experience applying role-based access control (RBAC) and data security best practices.
- Experience using Git, maintaining repositories, writing automated tests and building CI/CD workflows for data solutions.
- Experience developing data visualisations and reports.
- The ability to explain technical choices clearly, work collaboratively with business and technical stakeholders, and guide other engineers.
- PostgreSQL, SQL Server, Google BigQuery or Amazon Redshift.
- R or other programming languages.
- Relevant cloud services across Microsoft Azure, AWS or Google Cloud.
- Relevant certifications in Databricks, Snowflake or Microsoft Fabric.
- Flexible, hybrid working – 20 “work from anywhere” days per year on top of our hybrid policy.
- Global career pathways and non-linear progression; move between domains, disciplines or geographies.
- Continuous learning via certifications, internal academies, tech communities and funded conferences.
- Well-being & lifestyle allowance , loyalty leave, inclusive parental leave and employee assistance programmes.
- A vibrant, inclusive culture where diverse perspectives power innovation.
We look forward to exploring how your expertise can help our clients turn bold ideas into real-world impact.
- Monthly Lifestyle Allowance: Contribution towards health and wellbeing activities like gym memberships.
- Novated Leasing: Pre-tax car leasing benefit for new and used cars.
- Loyalty Leave: Receive an additional day of leave on your 3rd, 4th, and 5th work anniversaries, accumulating up to a maximum of 3 extra days of leave per year.
- Inclusive Parental Leave Policy: 12 weeks of primary carer’s leave and 4 weeks of secondary carer’s leave.
- Work From Anywhere: In addition to ourhybrid working policy, we also offer 20 days of working from anywhere per year. Ideal for an extended trip to get away from the city or visiting loved ones.
At Endava, we’re committed to creating an open, inclusive, and respectful environment where everyone feels safe, valued, and empowered to be their best. We welcome applications from people of all backgrounds, experiences, and perspectives—because we know that inclusive teams help us deliver smarter, more innovative solutions for our customers. Hiring decisions are based on merit, skills, qualifications, and potential. If you need adjustments or support during the recruitment process, please let us know.
Australian Citizens residing in Australia with Baseline Clearance only respond.
Role/s: EL1 Data Acquisition Engineer
The EL1 Data Engineer (Labour Hire) role will be a senior member of a project stream of the NDIA data warehouse platform. This includes updating data development documentation, hands-on data platform development, providing technical guidance to developers and reviewing/approving data designs and data models.
Key duties and responsibilities
- provide technical leadership to the project delivery team through design reviews, advising on implementation best practices, defining and refining release process and technical workshops.
- Review and refine the data delivery framework in order to enhance integration of new sources, performance of existing data delivery and rapid development of new subject areas.
- Own the technical implementation of new data from analysis to delivery (designing and delivering data dictionary, defining the release steps, defining best practices of data modelling and review steps).
- Facilitate continuous improvement measures in delivery principles, coding standards, documentation and provide training sessions to team.
- Provide knowledge transfer sessions and peer review on best practices on development, data modelling, release process etc.
- Prioritise work items and add them to a work queue.
- Understand, analyse and size user requirements.
- Development and maintenance of SQL analytical and ETL code.
- Development and maintenance of system documentation.
- Collaboration with data consumers, database development, testers and IT support teams.
(NOTE: the key responsibilities of the role are based on current priorities and may change over time)
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
