Databricks Application Engineer: Craft Scalable Pipelines

N2S.Global; Sydney; On-site; AUD 120,000 - 160,000; Full time

Job summary

N2S.Global in Australia is seeking a Databricks Application Engineer to design, develop, optimize, and support scalable data solutions on the Databricks platform. You will collaborate with data engineers, data scientists, and business stakeholders to deliver high-performance pipelines.

The role requires strong Spark and Databricks expertise, cloud experience, Delta Lake, and CI/CD practices. You will build notebooks, workflows, and automated jobs while monitoring data quality and governance.

Qualifications

• Bachelor's degree in CS, IT, engineering or related field.
• 3+ years of data engineering or big data experience.
• Hands-on Databricks experience.
• Strong programming in Python, PySpark or Scala.
• Strong SQL and data warehousing concepts.

Responsibilities

• Design, develop, optimize, and maintain scalable data pipelines on Databricks.
• Build and optimize ETL/ELT processes for large-scale data.
• Develop data ingestion from APIs, databases, files, and streaming sources.
• Implement Delta Lake architecture and data management best practices.
• Optimize Spark jobs for performance, scalability, and cost.
• Collaborate with business and technical teams to translate requirements into solutions.
• Develop notebooks, workflows, and automated jobs within Databricks.
• Monitor and troubleshoot production data pipelines and workflows.
• Implement data quality checks, governance, and security standards.
• Support CI/CD deployment processes and DevOps practices.
• Participate in code reviews and contribute to design discussions.
• Ensure compliance with data management policies and standards.

Skills

Databricks

PySpark

Python

SQL

Spark

ETL/ELT

Education

Bachelor's degree in Computer Science / IT / Engineering

Tools

Delta Lake

Unity Catalog

Databricks Workflows

Azure Databricks

Git CI/CD

Job description

N2S.Global in Australia is seeking a Databricks Application Engineer to design, develop, optimize, and support scalable data solutions on the Databricks platform. You will collaborate with data engineers, data scientists, and business stakeholders to deliver high-performance pipelines.

The role requires strong Spark and Databricks expertise, cloud experience, Delta Lake, and CI/CD practices. You will build notebooks, workflows, and automated jobs while monitoring data quality and governance.