Sr Data Engineer – Databricks & Power BI

SG Consulting Limited; Wellington; On-site; NZD 110,000 - 150,000; Full time

Job summary

SG Consulting Limited seeks a skilled Data Engineer to design and maintain scalable data pipelines using Databricks, while transforming data for analytics and BI. You will support Power BI reporting and collaborate with analysts to meet data needs.

Ideal candidates combine solid data engineering fundamentals with practical Power BI knowledge, delivering analytics-ready datasets and reliable data products across the business.

Qualifications

• Hands-on Databricks data engineering experience.
• Strong SQL and query optimization.
• Proficiency with Python and PySpark.
• Experience with ETL/ELT pipelines and data warehousing.
• Power BI report development and data modelling.
• Knowledge of Power Query and DAX.
• Experience with data quality checks and pipeline optimisation.
• Excellent analytical and communication skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build ingestion, transformation, and integration workflows.
• Develop and optimise SQL queries for processing and reporting.
• Work with Python and PySpark to transform large datasets.
• Implement ETL/ELT processes and ensure data quality.
• Develop and maintain data models for analytics and reporting.
• Work with Power BI to create and support reports and dashboards.
• Optimise Power BI data models and report performance.
• Collaborate with Data Scientists and stakeholders to define data needs.
• Implement data validation, monitoring, and documentation.

Skills

Databricks

SQL

Python

PySpark

ETL/ELT

Data warehousing

Data modelling

Power BI

Data quality

Troubleshooting

Communication

Tools

Azure Data Factory

Azure Data Lake Storage

Delta Lake

Spark

Power BI Service

Git

CI/CD

Power Query

DAX

Job description

We are seeking a skilled Data Engineer with hands-on experience in Databricks, data pipeline development, SQL, and Power BI. The successful candidate will be responsible for building and maintaining reliable data pipelines, transforming data for analytics and reporting, and supporting business intelligence solutions.

The ideal candidate will combine strong data engineering fundamentals with practical Power BI knowledge to deliver high-quality, accessible, and analytics-ready data.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Build data ingestion, transformation, and integration workflows.
• Develop and optimise SQL queries for data processing and reporting.
• Work with Python and PySpark to transform and process large datasets.
• Implement ETL/ELT processes and ensure data quality, accuracy, and consistency.
• Develop and maintain data models and curated datasets for analytics and reporting.
• Work with Power BI to create, enhance, and support reports and dashboards.
• Optimise Power BI data models, DAX calculations, and report performance.
• Collaborate with Data Scientists, Business Analysts, and stakeholders to understand data requirements.
• Implement data validation, monitoring, documentation, and performance optimisation.
• Follow data governance, security, and best practices for data management.

Required Skills and Experience

• Hands-on experience with Databricks and data engineering solutions.
• Strong SQL skills, including query optimisation and complex transformations.
• Experience with Python and PySpark for data processing.
• Understanding of ETL/ELT pipelines, data warehousing, and data modelling.
• Practical Power BI experience, including report development and data visualisation.
• Knowledge of Power Query, DAX, and Power BI data modelling.
• Experience with data quality checks, troubleshooting, and pipeline optimisation.
• Strong analytical, problem-solving, and communication skills.

Desirable Skills

• Experience with Azure Data Factory, Azure Data Lake Storage, or Microsoft Fabric.
• Knowledge of Delta Lake, Spark, and lakehouse architecture.
• Experience with CI/CD, Git, and automated data pipeline deployment.
• Familiarity with dimensional modelling and star schemas.
• Understanding of data governance, access controls, and cloud security.
• Experience with Power BI Service, workspace management, and deployment pipelines.

Location and Eligibility

Candidates based in Wellington or Christchurch are preferred. Applicants must have valid New Zealand working rights.