Senior Data Engineer

Datasmiths; Australia; Hybrid; AUD 140,000 - 200,000; Full time

Benefits offered by this job

Hybrid and remote work options

Competitive compensation

Flexible work culture

Growth and learning resources

Job summary

Datasmiths in Australia is seeking a Senior Data Engineer to design and optimize scalable data architectures and big data pipelines. You will work with cloud-native solutions, Databricks, Spark, and Delta Lake to deliver AI-powered insights.

The role emphasizes automation, governance, and collaboration with data scientists, analysts, and engineers to drive data-driven outcomes. Hybrid and remote options are available.

Qualifications

• 5+ years of experience in data engineering or cloud data solutions.
• Proficiency in Python, Scala, or SQL for building scalable data pipelines.
• Expertise in Databricks, Apache Spark, and Delta Lake.
• Experience with cloud platforms (AWS, Azure).
• Strong ETL, data lakes, and warehouse knowledge.
• Passion for automation, AI, and data workflows.

Responsibilities

• Build scalable data architectures – Design and implement high-performance pipelines.
• Optimise big data processing – Develop cost-effective data solutions using Databricks, Spark, and Delta Lake.
• Ensure data quality and governance – Implement data validation, lineage, and security.
• Drive automation and efficiency – Use Python, Scala, or SQL to automate workflows.
• Collaborate with cross-functional teams – Work with data scientists, analysts, and engineers.
• Stay ahead of the curve – Explore new data engineering trends, AI advancements, and cloud technologies.

Skills

Python

Scala

SQL

Databricks

Apache Spark

Delta Lake

Tools

Databricks

Apache Spark

Delta Lake

Job description

Location: Australia (Remote and Hybrid Options Available)

We’re looking for aSenior Data Engineerwho thrives on solving complex data challenges, designing robust architectures, and optimising big data pipelines. You’ll work withcutting-edge cloud and AI solutions, ensuring thatdata is accessible, scalable, and analytics-ready.

This role blendsdeep technical expertise, strategic problem-solving, and hands-on implementation. You’ll build and optimiseETL pipelines, data lakes, and data platformswhile collaborating with cross-functional teams to driveAI-powered insights.

Your Impact

• Build scalable data architectures– Design and implementhigh-performance data pipelines and cloud-native solutions.
• Optimise big data processing– Developefficient and cost-effective data solutionsusing Databricks, Apache Spark™, and Delta Lake.
• Ensure data quality and governance– Implementbest practices for data validation, lineage, and security.
• Drive automation and efficiency– UsePython, Scala, or SQLto automate data workflows and streamline processes.
• Collaborate with cross-functional teams– Work withdata scientists, analysts, and engineersto deliver data-driven solutions.
• Stay ahead of the curve– Continuously explore newdata engineering trends, AI advancements, and cloud technologies.

What We’re Looking For

• 5+ years of experienceindata engineering, big data, or cloud data solutions.
• Proficiency in Python, Scala, or SQLfor building scalable data pipelines.
• Expertise in Databricks, Apache Spark™, and Delta Lake.
• Experience with cloud platforms(AWS, Azure) and modern data architectures.
• Strong understanding of ETL processes, data lakes, and warehouse solutions.
• Excellent problem-solving skills– Able to tackle complex data challenges with innovative solutions.
• A passion for automation, AI, and optimising data workflows.

Nice to Have

• Databricks Certification(e.g., Databricks Certified Data Engineer or ML Associate).
• Experience withreal-time streaming architectures(Kafka, Kinesis, or similar).
• Knowledge ofAI/ML pipelines and MLOps best practices.
• Prior experience in aconsulting or client‑facing role.

Why Join Datasmiths?

• Be part of an AI-native company– We live and breathecutting-edge AI and data solutions.
• Work on high-impact projects– We partner withindustry leadersto solve some of the toughest data challenges.
• Shape the future of data engineering– Be at the forefront ofAI-driven transformation.
• Competitive compensation– We offera generous salary and performance-based bonuses.
• Flexible work culture– Hybrid and remote work options tofit your lifestyle.
• Grow with us– Accesstop-tier learning resources, training, and leadership development programs.

Our Commitment to Diversity & Inclusion

At Datasmiths,we celebrate diversity and welcome everyone– no matter your background, identity, or path into tech. We arecommitted to fostering an inclusive culturewhere everyonefeels valued, empowered, and inspired to thrive.

Ready to Engineer the Future of Data?

If you’re ready todesign next-gen data architectures, optimise big data solutions, and drive AI-powered insights, we want to hear from you.