Sr. Databricks Developer

Datamatics; Sydney, Melbourne; Híbrido; AUD 140.000 - 200.000; Jornada completa

Descripción de la vacante

Datamatics is seeking a Senior Databricks Developer to design, build and optimize scalable data pipelines using Databricks, Delta Lake and PySpark in a hybrid Sydney/Melbourne setup.

The role emphasizes data engineering, governance and collaboration with cross-functional teams to deliver reliable data products and actionable insights.

Formación

• 5+ years of experience in data engineering and platform development.
• Strong SQL and PySpark skills for scalable pipelines.
• Experience with Databricks, Delta Lake and Spark SQL.

Responsabilidades

• Design, develop and maintain scalable data pipelines using Databricks, Delta Lake and PySpark.
• Build efficient data transformation processes using SQL and Spark SQL.
• Develop robust data solutions for batch and operational processing.
• Apply software engineering best practices including reusable code, testing and automation.
• Implement unit and integration testing to ensure data quality.
• Manage source code, branching and deployments using Git and CI/CD.
• Support production releases and ensure readiness.
• Monitor platform health with observability and alerting capabilities.

Conocimientos

Databricks

Delta Lake

PySpark

SQL

Spark SQL

Data pipelines

CI/CD practices

Git

Observability

Unity Catalog

Herramientas

Git

CI/CD

Workflows

Jobs

Unity Catalog

Descripción del empleo

Role : Senior Databricks Developer

Location: Sydney/ Melbourne

Work Mode: Hybrid

Experience required: 5+yrs

Requirement:

Data Engineering & Platform Development

• Design, develop and maintain scalable, reliable data pipelines using Databricks, Delta Lake and PySpark.
• Build efficient and optimized data transformation processes using SQL and Spark SQL.
• Develop robust data solutions that support both batch and operational data processing requirements.
• Apply software engineering best practices to data engineering delivery, including reusable code, testing and automation.

Quality Engineering & DevOps

• Implement unit testing and integration testing to ensure data quality, reliability and maintainability.
• Manage source code, branching strategies and deployments using Git and CI/CD practices.
• Support production releases and ensure solutions meet operational and production readiness standards.
• Monitor platform health through observability, monitoring and alerting capabilities, proactively identifying and resolving issues.

Data Platform & Governance

• Deliver data assets through the full lifecycle, including discovery, design, development, testing, deployment and operational support.
• Adhere to engineering standards, code review processes and governance controls.
• Optimize and continuously improve data pipelines and platform performance.
• Utilize Databricks platform capabilities, including Workflows, Jobs and Unity Catalog, to support secure and governed data delivery.

Agile Delivery

• Work effectively within Agile delivery teams, participating in planning, estimation, sprint ceremonies and continuous improvement activities.
• Contribute to delivery outcomes while balancing feature development, operational support and platform enhancements.

Stakeholder & Business Engagement

• Collaborate with business and technical stakeholders to understand, clarify and refine requirements.
• Translate business problems and data needs into practical technical solutions and implementation plans.
• Support prioritization discussions and contribute to delivering business value through data products and services.

Collaboration & Communication

• Work collaboratively within cross-functional teams comprising engineers, product owners, analysts and business stakeholders.
• Produce clear and effective technical documentation, including solution designs, implementation approaches and operational procedures.
• Participate in peer reviews, knowledge sharing sessions and collaborative problem-solving activities.

Continuous Improvement & Capability Uplift

• Demonstrate a commitment to continuous learning and keeping current with modern data engineering practices and technologies.
• Contribute to uplifting team capability through mentoring, coaching and sharing best practices.
• Support continuous improvement initiatives that enhance engineering maturity, delivery efficiency and platform capability.