Senior Data Engineer

TL Consulting Group; Sydney; Hybrid; AUD 150,000 - 210,000; Full time

Benefits offered by this job

Competitive salary

Hybrid work model

Professional development

Job summary

TL Consulting Group is seeking a Senior Data Engineer to design, build, and optimize a scalable data platform across Azure Databricks and Microsoft Fabric. You will mentor data engineers, establish data architecture standards, and collaborate with scientists, analysts, and business stakeholders to deliver data-driven outcomes.

The role is Sydney-based, hybrid, and involves ensuring data quality, governance, and cost-efficient processing for enterprise-scale solutions across BFSI, government,

Qualifications

• Bachelor’s degree in engineering, CS, or related field.
• 5+ years in data engineering roles.
• 3+ years in consulting or professional services.

Responsibilities

• Design, build, and optimize data platform across Azure Databricks and Fabric.
• Mentor junior/mid-level data engineers on best practices.
• Lead data architecture standards and governance policies.
• Collaborate with data scientists, analysts, and stakeholders to translate requirements.
• Monitor and optimize performance and cost of pipelines.

Skills

Azure Databricks

Microsoft Fabric

Delta Lake

Python

SQL

ETL/ELT

Education

Bachelor’s degree in engineering or Computer Science

Tools

Databricks

OneLake

Delta Lake

Terraform

Job description

At TL Consulting, we simplify modern complexity with advanced, sustainable engineering. As a proudly Australian-owned consultancy with over 15 years of experience, we empower businesses to innovate and achieve measurable outcomes through scalable, cloud-native solutions. Our 50+ consultants operate across key sectors including banking, financial services & insurance (BFSI), government, retail, education, and entertainment. This cross-industry expertise allows us to deliver customer-centric solutions that combine deep technical capability with a clear understanding of each client’s strategic vision. Our approach leverages proven, scalable blueprints tailored to your business direction - enabling accelerated outcomes, maximized ROI, and sustainable long-term adoption.

We maintain strategic partnerships with organizations that share our vision for a cloud-native future. Through our collaborations with Microsoft, GitHub, and Databricks, we ensure our capabilities stay aligned with evolving best practices. These partnerships also provide access to innovation programs, co-funding opportunities, and dedicated support - helping us accelerate time-to-value for our clients.

About the role

We are seeking an experienced Senior Data Engineer to design, build, and optimize modern data platform across Azure Databricks and Fabrics. This role will be instrumental in architecting and developing scalable data solutions using Databricks and Microsoft Fabric, enabling data-driven decision-making across the organization. A key role will be mentoring Data engineers.

Key Responsibilities

Data Platform Development

• Design and implement robust data pipelines and ETL/ELT workflows using Databricks and Microsoft Fabric
• Architect and maintain lakehouse solutions leveraging Delta Lake and OneLake architectures
• Optimize data processing workflows for performance, cost-efficiency, and scalability
• Develop and maintain real-time and batch data integration solutions

Technical Leadership

• Lead the implementation of data architecture standards and best practices
• Mentor junior and mid-level data engineers on modern data engineering techniques
• Collaborate with data scientists, analysts, and business stakeholders to understand requirements
• Evaluate and recommend new technologies and tools to enhance the data platform

Data Quality & Governance

• Implement data quality frameworks and monitoring solutions
• Establish and enforce data governance policies and security controls
• Ensure compliance with data privacy regulations and organizational standards
• Design and implement data lineage and metadata management solutions

Performance & Optimization

• Monitor and optimize query performance across Databricks and Fabric environments
• Implement cost optimization strategies for cloud data infrastructure
• Troubleshoot complex data pipeline issues and implement solutions

Skills & Experience

Education: Bachelor’s degree in engineering, Computer Science, or a related field. Master’s degree or MBA preferred.

Experience:

• 5+ years of experience in data engineering roles
• 3+ years of experience in a consulting or professional services environment.
• Proven experience in leading data engineering teams and delivering client-focused solutions.

Technical Skills:

• Expertise in data analytics tools and platforms (e.g., SQL, Python, R, Apache Spark, Tableau, Power BI).
• Expertise in Azure services and analytics platforms such as Databricks, Synapse and MS Fabric.
• Strong expertise with Databricks platform including Delta Lake, Spark, and MLflow
• Hands-on experience with Microsoft Fabric (Data Factory, Synapse, Power BI, OneLake)
• Proficiency in Python and/or Scala for data processing
• Advanced SQL skills and experience with query optimization
• Experience with other cloud platforms (AWS, or GCP) is a plus
• Knowledge of infrastructure as code (Terraform, ARM templates) would be a plus
• Has worked with Databricks Assets Bundles
• Familiarity with CI/CD pipelines for data workflows
• Certifications in Databricks and/or Microsoft Azure
• Strong understanding of data warehousing concepts and dimensional modeling

Soft Skills:

• Strong client focus and ensures effective relationships are in place with key client stakeholders to deliver that service.
• Excellent client relationship management skills with the ability to explain complex technical concepts to non-technical stakeholders.
• Solid experience in relationship management, conflict resolution, influencing and negotiation.
• Must be able to work independently but also lead Consulting engagements in a highly matrixed environment (especially for short-term engagements).
• A team player who can liaise effectively with a diverse group of delivery teams to ensure a positive experience for the customer.
• Ability to manage multiple projects simultaneously and prioritize effectively in a dynamic environment.
• Excellent verbal and written communication skills.

Preferred Qualifications:

• Consulting experience in various industries (e.g., finance, healthcare, manufacturing).
• Azure and Databricks certifications are a plus.
• Experience with Agile methodologies and project management tools.

WHY JOIN US

• Be part of a dynamic consultancy firm that values innovation, growth, and professional development.
• Lead a growing practice at the forefront of data-driven business transformation.
• Work with diverse, top-tier clients across multiple industries, delivering impactful solutions.
• Competitive salary and benefits package.
• Career development and growth opportunities within a rapidly expanding organization.

This is a Sydney based, permanent opportunity and will be hybrid in nature. No agencies please. Candidates must have valid visa with full work rights in Australia. Visas will not be transferred or sponsored.

TL Consulting Group understands the importance of fostering a diverse and inclusive workplace. We are committed to ensuring that everyone, no matter their gender, identity, sexual orientation, race, ethnicity, religion, disability, or age, has equal access to opportunities. We warmly welcome applications from all individuals, including Aboriginal and Torres Strait Islander peoples, members of the LGBTQI+ community, and individuals with disabilities.