Etl Test Lead - Azure Synapse Data Warehouse

Cognizant; Sydney; Hybrid; AUD 120,000 - 170,000; Full time

Job summary

NTT DATA is seeking an experienced Senior Data Engineer to design, build and optimize modern cloud data platforms. You will create scalable data pipelines, data models and lakehouse solutions using Microsoft Azure and Fabric, collaborating with architects, analysts and stakeholders to deliver data-driven outcomes.

The role emphasizes hands-on SQL and Python development, data governance and cost-efficient operations, with a focus on AI-enabled analytics for enterprise customers.

Qualifications

• 5+ years of Data Engineering, Data Warehousing or Cloud Data Platform development.
• Strong experience with Microsoft Azure data services: Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure SQL, Azure Databricks, Azure DevOps.
• Hands-on with Microsoft Fabric: Data Factory, Lakehouse, Warehouse, OneLake, Power BI, Notebooks.
• Strong SQL development and data modelling; experience with Python/PySpark.
• Experience in Agile delivery environments and stakeholder engagement.

Responsibilities

• Design and develop scalable data pipelines and ELT/ETL processes.
• Build and maintain data platforms using Microsoft Fabric and Azure services.
• Develop Lakehouse, Data Warehouse and Data Factory solutions in Microsoft Fabric.
• Implement data integration across cloud and on‑prem sources.
• Design and optimize data models to support analytics, reporting and AI initiatives.
• Develop data transformations using SQL, Spark and Python.
• Implement data quality, governance and security controls.
• Monitor and optimize platform performance, reliability and cost.

Skills

Data engineering

SQL development

Python/PySpark

Agile delivery

Stakeholder engagement

Tools

Azure Data Factory

Azure Synapse Analytics

Azure Databricks

Azure Data Lake Storage

Microsoft Fabric

Power BI

Python

Job description

• What Etl Test Lead Azure Synapse Data Warehous...

Showing 1059 Etl Test Lead Azure Synapse Data Warehouse jobs in Sydney

**Make an impact with NTT DATA**
Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion - it's a place where you can grow, belong and thrive.
We are seeking an experienced Data Engineer to join our Data & AI practice. You will design, build and optimize modern cloud data platforms, enabling organizations to unlock value from data through scalable analytics, reporting and AI solutions.
Working with a range of enterprise customers, you will develop robust data pipelines, data models and lakehouse solutions using Microsoft Azure and Microsoft Fabric technologies.

Key Responsibilities

• Design and develop scalable data pipelines and ELT/ETL processes.
• Build and maintain data platforms using Microsoft Fabric and Azure services.
• Develop Lakehouse, Data Warehouse and Data Factory solutions in Microsoft Fabric.
• Implement data integration solutions across cloud and on-premises data sources.
• Design and optimize data models to support analytics, reporting and AI initiatives.
• Develop and maintain data transformation processes using SQL, Spark and Python.
• Implement data quality, governance and security controls.
• Monitor and optimize platform performance, reliability and cost efficiency.
• Collaborate with architects, analysts, data scientists and business stakeholders to deliver data-driven outcomes.
• Support CI/CD implementation and automated deployment practices.

Skills & Experience

Essential

• 5+ years' experience in Data Engineering, Data Warehousing or Cloud Data Platform development.
• Strong experience with Microsoft Azure data services including:
• Azure Data Factory
• Azure Data Lake Storage
• Azure Synapse Analytics
• Azure SQL
• Azure Databricks
• Azure DevOps
• Hands-on experience with Microsoft Fabric including:
• Data Factory
• Lakehouse
• Warehouse
• OneLake
• Power BI
• Notebooks
• Strong SQL development and data modelling skills.
• Experience developing data pipelines using Python and/or PySpark.
• Understanding of data warehousing concepts including star and snowflake schemas.
• Experience working within Agile delivery environments.
• Strong stakeholder engagement and problem-solving skills.

Desirable

• Experience with Snowflake data platforms.
• Experience with Databricks Lakehouse architecture.
• Experience implementing data governance solutions such as Microsoft Purview
• Exposure to real-time or streaming data architectures.
• Experience working in consulting or professional services environments.

AI & Emerging Technologies (Advantageous)

• Experience integrating AI and Generative AI capabilities into data platforms.
• Knowledge of Azure AI Services, Azure OpenAI, Microsoft Copilot Studio or Microsoft Fabric AI capabilities.
• Experience supporting data foundations for machine learning or AI workloads.
• Understanding of vector databases, Retrieval-Augmented Generation (RAG) and modern AI architectures.

Qualifications

• Microsoft Azure certifications highly regarded, such as:
• Azure Data Engineer Associate (DP-203)
• Fabric Analytics Engineer Associate (DP-600)
• Azure Fundamentals (AZ-900)

About NTT DATA

NTT DATA is a $30+ billion business and technology services leader, serving 75% of the Fortune
Global 100. We are committed to accelerating client success and positively impacting society through
responsible innovation. We are one of the world's leading AI and digital infrastructure providers, with
unmatched capabilities in enterprise-scale AI, cloud, security, connectivity, data centers and
application services. Our consulting and industry solutions help organizations and society move
confidently and sustainably into the digital future. As a Global Top Employer, we have experts in more
than 70 countries. We also offer clients access to a robust ecosystem of innovation centers as well as
established and start-up partners. NTT DATA is part of NTT Group, which invests over $3 billion each
year in R&D.

Equal Opportunity Employer

NTT DATA is proud to be an Equal Opportunity Employer with a global culture that embraces diversity. We are committed to providing an environment free of unfair discrimination and harassment. We do not discriminate based on age, race, colour, gender, sexual orientation, religion, nationality, disability, pregnancy, marital status, veteran status, or any other protected category. Join our growing global team and accelerate your career with us. Apply today.

As a Senior Data engineer with expertise in software development / programming and a passion for building data-driven solutions, you're ahead of trends and work at the forefront of Big Data and Data warehouse technologies.

We are seeking people who are:

• Passionate about building next generation data platforms and data pipeline solution across the bank.
• Ready to execute state-of-the-art coding practices, driving high quality outcomes to solve core business objectives and minimise risks.
• Capable to create both technology blueprints and engineering roadmaps, for a multi- year data transformational journey.
• Are experienced in providing data driven solutions that source data from various enterprise data platform into Cloudera Hadoop Big Data environment, using technologies like Spark, MapReduce, Hive, Sqoop, Kafka; transform and process the source data to produce data
  assets; and transform and egression to other data platforms like Teradata or RDBMS system.
• Are experienced in building effective and efficient Big Data and Data Warehouse frameworks, capabilities, and features, using common programming language (Scala, Java, or Python), with proper data quality assurance and security controls.
• Are experienced in designing, building, and delivering optimised enterprise-wide data ingestion, data integration and data pipeline solutions for Big Data and Data warehouse platforms.
• Are confident in building group data products or data assets from scratch, by integrating large sets of data derived from hundreds of internal and external sources.
• Can lead and mentor other data engineers in a project work or initiative.
• Have experience and responsible for data security and data management.

Technical skills

Experience in designing, building, and delivering enterprise-wide data ingestion, data integration and data pipeline solutions using common programming language (Scala, Java, or Python) in a Big Data and Data Warehouse platform. Preferably with at least 5+ years of hands‑on experience in a Data Engineering role.

Experience in building data solution in Hadoop platform, using Spark, MapReduce, Sqoop, Kafka and various ETL frameworks for distributed data storage and processing. Preferably with at least 5+ years of hands‑on experience.

Strong Unix/Linux Shell scripting and programming skills in Scala, Java, or Python.

Proficient in SQL scripting, writing complex SQLs for building data pipelines.

Experience in leading and mentoring data engineers, including ownership of internal business stakeholder relationships and working with consultants.

Experience in working in Agile teams, including working closely with internal business stakeholders.

Familiarity with data warehousing and/or data mart build experience in Teradata, Oracle or RDBMS system is a plus.

Certification on Cloudera CDP, Hadoop, Spark, Teradata, AWS, Ab Initio is a plus.

Experience in Ab Initio software products (GDE, Co>Operating System, Express>It, etc.) is a plus.

Experience in AWS technology (EMR, Redshift, DocumentDB, S3, etc.) is a plus.

You are passionate to stay ahead of the latest AWS Cloud, Gen AI, Data Lake, and Data Mesh technologies.

We are one of the largest and most advanced Data Engineering teams in the country.

Together we can build state-of-the-art data solutions that power seamless experiences for millions of customers.

Do work that matters:

As a Staff Data Engineer with expertise in software development / programming and a passion for building data-driven solutions, you are ahead of trends and work at the forefront of AWS Cloud, Data Mesh and AI technologies.

See yourself in our team:

Join the Business Banking (BB) Technology team at CommBank, where we are driving innovation through Digital and Gen AI platforms that empower our Business Bank customers.

Our team plays a pivotal role in delivering secure, reliable, and actionable data products. These assets support strategic decision-making, risk management, and customer success. This is driven by critical data assets which is supported by the Data Engineering teams within the BB Technology who are responsible for ingesting data, creating ESDM aligned Data Products and creating reports which will help business produce strategies.

We are also interested in hearing from people who:

Think AI-first and are passionate about solving problems with intelligent data solutions.

Have a strong learning mindset and thrive in collaborative environments.

Champion a culture of quality, openness, and continuous improvement.

Design and build next-gen data platforms and pipelines using Python, PySpark, or Scala.

Integrate data from diverse sources into centralised lakes and cloud platforms.

Create technology blueprints and engineering roadmaps for long-term transformation.

Ensure data security, governance, and compliance.

Deliver end‑to‑end solutions that meet business needs and minimise risk.

Build high‑quality, scalable data products from scratch.

Communicate effectively with stakeholders and contribute to team‑wide engineering practices.

Define and evolve shared semantic models and metric definitions to ensure consistent meaning and reusable data across products and domains.

Drive adoption of catalogues, glossaries, and stewardship practices that promote discoverability, accountability, and self‑service.

Embed policy‑as‑code for data access, classification, and retention ensuring compliance with internal and regulatory standards.

Implement automated testing, observability, and end‑to‑end lineage to improve data trust, traceability, and incident resolution speed.

Key Skills & Experience:

AWS Certified Data Engineer Associate with demonstrable experience in Data Engineering.

Data Processing: Strong expertise in Python, PySpark, SQL, Spark, Scala, and distributed data processing.

Experience in AWS data platforms (Glue, Lambda, Redshift, Athena, Kinesis, EMR).

Data Governance & Security: Experience with AWS DataZone, Lake Formation, IAM, encryption.

Experience in using AI Powered coding Assistants like Cline, Roo and Aider.

Marketplace & APIs: Exposure to AWS Marketplace, API Gateway, and data monetization strategies.

Automation & CI/CD: Experience with Terraform, CloudFormation, GitOps, and serverless frameworks.

Performance Optimisation: Ability to fine‑tune ETL jobs, query performance, and cost efficiencies.

Experience with Snowflake, or Apache Iceberg.

Knowledge of data product thinking and data mesh principles.

Working with us

Whether you are passionate about customer service, driven by data, or called by creativity, a career with CommBank is for you.

We support our people with the flexibility to balance where work is done with at least half your time each month connecting in our Sydney or Melbourne office. We also have many other flexible working options available including changing start and finish times, part‑time arrangements and job share to name a few.

Technology Founder, AI‑Enabled Product & Data Engineering

The deal, up front. Most founder conversations start with two questions: how do we pay ourselves, and how do we raise. Both are settled before you begin.

A$3m of committed capital per venture, roughly two years of focused runway.

30% equity for the founding team. Genuine founder ownership, not an employee option grant, allocated across however many founders the venture calls for.

You control the budget. Founder salaries are paid from the A$3m, and the founders set them, alongside every other cost, in a business plan Repeat Builders does not write for you.

The support is the point. Every venture runs on the AI‑native operating system we've built, the playbooks and tooling behind each company. That includes the talent system for your first hires. It also includes the growth analytics engine that validated your demand. Our team works alongside yours on product and hiring, and helps with strategy. You'll work from a shared office at Tech Central, with other founders and investors around you. We're most involved at the start, and step back as the company builds its own capability.

The two‑year arc. We take the idea to a validated MVP. You take it from there: year one to product in market and measurable traction, year two to raising the next round on its own strength and running independently of us. Success is measured by traction, not activity.

The opportunity

You'll be the technical co‑founder of one of our ventures. It arrives with a working product, real users and the demand data behind it (reach, conversion, cost per signup and a customer email list), and it becomes yours to build. It also arrives with the tree: every variation of the idea we scored on the way in, so a new feature or a pivot starts from evidence.

About 40 prototypes sit in the pipeline today, from claims and compliance tools to consumer health apps, and some need real depth in satellite imagery or computer vision. Many of the ventures we build run on data that isn't theirs: financial, property or location data, drawn from sources you don't control and have to make reliable. Others depend on AI that must be right, and sometimes has to answer in real time. Your job is to turn that into product people rely on, with a co‑founder alongside you and Repeat Builders behind you all.

What you will own

• Ship a working product surface fast enough to get real market feedback inside the first sprint cycle.
• Build the data layer the product depends on: ingesting, cleaning and joining third‑party data well enough that customers can trust what the product tells them.
• Make AI work in production, including where it has to respond in real time and where a wrong answer costs the customer.
• Make the build, buy or AI‑assist call on every significant technical decision, including the harder judgement of when an AI‑assisted path is faster in practice and when it only looks faster.
• Stand up the engineering foundation a small team runs on, from the repo to CI/CD, before anyone else is there to set it.
• Turn validated demand into shipped, tested product, working with your co‑founder and Repeat Builders and adjusting as user feedback replaces assumption.
• Hire and lead the first engineers as the company grows past what one founder can build.

What you will bring: essential

• Full‑stack ability, proven by product you have shipped and people use. You built it somewhere demanding, a fast‑growing startup or a strong engineering team, whatever the size or brand of the company.
• Data engineering in production. Pipelines, integrations or ingestion you built and kept running, ideally on data you didn't control. Most of our ventures live or die on getting this right.
• AI has changed how you build, not just how you talk about it. You reach for coding and agent tools daily, and you know where they earn their keep and where they get in the way.
• A significant technical or product decision you made on incomplete information and owned through to its outcome.

Non‑essential, specialist skills

These strengthen an application. None is required.

• Real‑time or low‑latency systems: streaming, voice, or anything where response time is part of the product.
• LLM features shipped to real users, including the evaluation and guardrails that keep them honest.
• Running what you've built: reliability, on‑call, and the unglamorous work of keeping a system up once people depend on it.
• Experience in a regulated or data‑sensitive domain, such as finance, compliance or property, where accuracy is the product.
• Geospatial or computer vision, for the ventures that read satellite or camera imagery.

If you clear most of this and the rest of this ad speaks to you, apply. The things that make you unusual belong in the application, not on a checklist.

About Repeat Builders

Repeat Builders is a Sydney venture builder launching its first five ventures over the next six months, each on a fixed, funded runway.

Anyone can vibe code an app now. The hard part is knowing whether anyone wants it. Every idea we build is scored and tested with real users before a founder is engaged, with over 2,200 scored so far. You inherit the demand data and the tree of related ideas, and the growth analytics engine behind them stays with you.

That's the first layer. The second is harder for an investor to copy. We work in the same space, on the same AI‑native systems, and you'd be one of five ventures. The aim for each is to stand on its own after two years.

We’re interested in hearing from people who:

• Have solid hands‑on experience of data engineering/ETL principles and practices with
• Unix/Linux, SQL, Ab Initio and other data integration tools and frameworks.
• Are confident demonstrating coding, design, debugging and problem‑solving skills
• Have the ability to mentor and provide technical assistance to team members.
• Mindset to apply and promote industry best patterns and practices.
• Experience in developing and deploying high‑quality software solutions with comprehensive
• test coverage without supervision.
• Experience with estimating development effort for new features
• Ability to lead Level 3 or above support and technical troubleshooting activity.
• We use a broad range of tools, languages, and frameworks. We don’t expect you to know them
• all but experience or exposure with some of these (or equivalents) will set you up for success in
• this team.
• Demonstrated hands‑on programming experience in Ab Initio GDE and related products and
• shell scripting. Background with both Batch and continuous flows is highly regarded.
• Experience with RDBMS (Oracle) concepts and using SQL, Stored Procedures and packages.
• Experience working on real‑time applications using either MQs, REST APIs, TCP or Kafka.
• Exposure to Agile working environment and DevOps roles.
• CI/CD concepts and ability to create pipelines in Jenkins, TeamCity, Ab Initio EME or Github
• Actions.
• Coding with Python or Java preferred, not a must have.