Senior Data Engineer: Scalable Pipelines & Cloud Platform

The Leaves, Inc.; Australia; Vor Ort; AUD 180.000 - 240.000; Vollzeit

Zusammenfassung

The Leaves, Inc. is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines and modern data platforms. You will integrate structured and unstructured data from multiple sources, develop data warehouses and ensure data quality across large-scale workloads.

You will collaborate with data scientists, BI teams and software engineers to support reporting and analytics, while optimising SQL performance and ensuring governance and security across cloud environments.

Qualifikationen

• Bachelor's or Master's degree in a related technical discipline.
• 10-20 years of relevant professional experience in data engineering, data platform development or related fields.
• Strong hands-on experience designing production-grade data pipelines and enterprise data solutions.
• Experience working with cloud platforms, data warehouses, ETL/ELT frameworks and large-scale data processing systems.
• Demonstrated ability to troubleshoot complex technical challenges and optimise data infrastructure.
• Experience in enterprise environments, large-scale projects or cross-functional teams is advantageous.

Aufgaben

• Design, develop and maintain scalable data pipelines and data processing systems.
• Build and optimise ETL/ELT workflows for extracting, transforming and loading data from multiple sources.
• Develop and maintain data warehouses, data lakes and modern data platforms.
• Integrate structured and unstructured data from internal and external systems.
• Design, implement and optimise database schemas, data models and data storage solutions.
• Work with large datasets to ensure data accuracy, consistency, availability and reliability.
• Develop batch processing and real-time data streaming solutions.
• Implement data quality checks, validation processes and monitoring mechanisms.
• Collaborate with data scientists, business analysts, software engineers and BI teams to support reporting and analytics requirements.
• Optimise SQL queries, database performance and data processing workloads.
• Implement data security, access controls, governance and compliance measures.
• Deploy and maintain cloud-based data solutions using AWS, Azure or Google Cloud Platform.
• Work with distributed processing technologies and big data frameworks where required.
• Identify and resolve data pipeline failures, performance bottlenecks and integration issues.
• Automate data workflows, testing, deployment and operational monitoring.
• Maintain technical documentation, data dictionaries, pipeline specifications and architecture diagrams.

Kenntnisse

Data pipeline design

Cloud platforms experience

ETL/ELT frameworks

SQL performance tuning

Big data processing

Collaborative teamwork

Ausbildung

Bachelor's degree in CS/IT/DE/SE

Master's degree in CS/IT/DE/SE

Tools

AWS

Azure

GCP

Hadoop/Spark

Jobbeschreibung

The Leaves, Inc. is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines and modern data platforms. You will integrate structured and unstructured data from multiple sources, develop data warehouses and ensure data quality across large-scale workloads.

You will collaborate with data scientists, BI teams and software engineers to support reporting and analytics, while optimising SQL performance and ensuring governance and security across cloud environments.