Senior Data Engineer

Tata Consultancy Services; Auckland; On-site; NZD 120,000 - 180,000; Full time

Benefits offered by this job

Competitive salary

Global exposure

Structured learning and development

Career progression

Job summary

Tata Consultancy Services in Australia is seeking an experienced Data Engineer to design scalable data platform architectures on Azure and Databricks, implement medallion patterns, and craft reliable data pipelines for analytics and reporting.

You will lead legacy modernization, enforce data quality and governance, model data for analytical workloads, and collaborate with stakeholders to deliver end‑to‑end solutions while applying CI/CD practices and mentoring junior engineers.

Qualifications

• 8+ years of experience in data engineering, data platform development, or data architecture roles.
• Deep expertise in Databricks (Delta Lake, medallion architecture, Unity Catalog) and Azure Data Services (ADF, ADLS, Synapse, Azure SQL).
• Strong SQL, PySpark/Spark, and Python skills with hands-on pipeline development experience at scale.
• Proven experience designing and operating data pipelines at enterprise scale - batch and streaming.
• Solid understanding of data modelling, data governance, and data quality frameworks.
• Experience with CI/CD for data pipelines using Azure DevOps, including automated testing and release management.

Responsibilities

• Design and implement scalable data platform architectures on Azure and Databricks, including medallion patterns and lakehouse architectures.
• Build and maintain robust ETL/ELT pipelines using Databricks and Azure data services for batch and streaming workloads.
• Lead legacy data pipeline migrations to cloud-native solutions on Azure/Databricks with migration strategies and cutover plans.
• Establish data quality frameworks, lineage tracking, and access governance aligned with policies.
• Create logical and physical data models for analytic and operational workloads, optimizing for performance and cost.
• Engage with client data teams, present architectures, and support estimation and planning.
• Develop CI/CD pipelines and automated testing for data workloads; mentor engineers across teams.

Skills

Data engineering

Databricks

Azure Data Services

SQL

PySpark/Spark

Data modelling

CI/CD for data

Stakeholder communication

Tools

Azure DevOps

Job description

Join Tata Consultancy Services, Asia Pacific and be part of an organization committed to sustainable development for our future. TCS follows the Tata group philosophy of building sustainable businesses that are rooted in the community and demonstrate care for the environment. Our unique values position us to combine a purpose-driven worldview with digital innovation, collaborating with customers, communities and governments to lead and shape innovative solutions for a sustainable future. TCS has been carbon neutral in its operations across 11 countries, 12 delivery centres and 18 offices in Asia Pacific since 2022. This is only the initial stage in TCS' journey as we strive to achieve long-term net zero emissions by 2030.

Corporate sustainability is embedded in our triple-bottom-line, focusing on people, the planet, and our purpose. Our offices are designed with eco-friendly features that significantly reduce our carbon footprint and enhance energy efficiency. We actively champion green initiatives, such as promoting paperless operations, implementing energy-efficient practices, and fostering employee engagement in sustainability efforts. When you become part of the TCS family, you will play an essential role dedicated to innovation, excellence, and crafting a brighter, greener future together. Join us and be a part of our mission to drive sustainability through technology and talent at Tata Consultancy Services, APAC today.

Roles and Responsibilities:

• Data Platform Architecture: Design and implement scalable data platform architectures on Azure and Databricks, including medallion architecture (Bronze/Silver/Gold), data lakehouse patterns, and integration layers that serve analytics, reporting, and downstream consumers.
• Pipeline Development: Build and maintain robust ETL/ELT pipelines using Databricks (PySpark/Spark SQL), and related Azure data services. Ensure pipelines handle batch and streaming workloads reliably at scale.
• Legacy Modernisation: Lead the migration of legacy data pipelines to modern cloud-native solutions on Azure and Databricks. Define migration strategies, manage cutover plans, and ensure zero data loss during transition.
• Data Quality & Governance: Establish and enforce data quality frameworks, validation rules, and monitoring across all pipelines. Implement data lineage tracking, cataloguing (Unity Catalog), and access governance in line with organisational policies.
• Data Modelling & Design: Design logical and physical data models for analytical and operational workloads. Define schemas, partitioning strategies, and optimisation patterns for performance and cost efficiency across Delta Lake and Azure SQL.
• Stakeholder Engagement: Engage with the client’s Data & Analytics team on technical discussions, solution walkthroughs, and progress updates. Participate in requirement workshops and provide technical input to support estimation and planning.
• DevOps & Automation: Build and maintain CI/CD pipelines for data workloads using Azure DevOps. Implement automated testing for data pipelines, and deployment automation across environments.
• Mentorship & Standards: Review code, enforce engineering standards, and mentor offshore and onsite data engineers. Drive adoption of best practices for Spark, SQL, Python, testing, and version control across the team.

Key Skills:

• 8+ years of experience in data engineering, data platform development, or data architecture roles
• Deep expertise in Databricks (Delta Lake, medallion architecture, Unity Catalog) and Azure Data Services (ADF, ADLS, Synapse, Azure SQL)
• Strong SQL, PySpark/Spark, and Python skills with hands‑on pipeline development experience at scale
• Proven experience designing and operating data pipelines at enterprise scale - batch and streaming
• Solid understanding of data modelling (dimensional, data vault, or lakehouse patterns), data governance, and data quality frameworks
• Experience with CI/CD for data pipelines using Azure DevOps, including automated testing and release management
• Track record of leading legacy‑to‑cloud data migration programmes (SSIS, on‑premise SQL to Azure/Databricks)
• Excellent communication skills — able to present technical concepts to both engineering teams and senior stakeholders

Top Reasons to Join Our Team at TCS Australia

Attractive compensation & rewards

We offer a highly competitive salary package aligned with the Australian market, complemented by performance-based incentives, recognition programs, and comprehensive employee benefits. Your achievements are celebrated, and your contributions truly matter.

Global exposure with local impact

At TCS, you'll work with leading Australian enterprises across diverse industries while collaborating with our global network of experts. This means access to world-class practices, transformative technologies, and the chance to shape solutions that have a real impact - locally and globally.

Empowering leadership & supportive environment

Our leadership team is committed to helping you succeed. We encourage initiative, reward innovation, and provide a collaborative environment where your ideas are heard and valued.

We believe in careers, not just jobs. With continuous learning, certification support, career mobility, and clear progression pathways, TCS empowers you to grow and evolve - whether you want to deepen your expertise, explore new domains, or take on leadership roles.

Inclusive, collaborative culture

Diversity, inclusion, and respect are central to who we are. At TCS Australia, you'll join a workplace where collaboration thrives, individuality is valued, and everyone can contribute to meaningful outcomes.

Continuous learning & cutting‑edge innovation

We invest heavily in your development, offering structured learning programs, global certifications, leadership development, and opportunities to work with emerging technologies. You'll be part of an environment that constantly challenges and expands your capabilities.

A Great Fit for the TCS Culture

We're looking for people who are:

• Passionate about technology and transformation
• Driven by curiosity, innovation, and continuous learning
• Excited to work in a collaborative, purpose‑driven environment
• Motivated by delivering real impact for clients, communities, and society

The TCS Community in Australia and Beyond

• Global strength: Over 600,000+ associates in 55+ countries, united by a shared purpose — to innovate and make a difference.
• Local presence: A trusted partner to Australia’s leading enterprises across financial services, government, retail, healthcare, telecom, and more.
• Social impact: Deeply committed to sustainability, digital inclusion, and community engagement initiatives in Australia and around the world.
• Empowered workplace: We foster a culture where diversity, equity, and accessibility are integral to how we operate.

We don't just imagine a better future - we help build it.

About TCS

Tata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. We have partnered with many of the world's largest organisations to help them transform their businesses, operating models, and technology ecosystems.

Learn more at www.tcs.com.

Equal Opportunity Employer

TCS is an equal opportunity employer. All qualified applicants will be considered without regard to race, colour, religion, gender, gender identity, sexual orientation, national origin, disability, veteran status, or any other characteristic protected by applicable law.