Lead Data Platform Reliability Engineer

Ukg; Sydney; Sur place; AUD 120 000 - 170 000; Plein temps

Résumé du poste

IMC in Sydney seeks a Data Engineer to own large-scale data collection, storage, access and query platforms, ensuring safe operation and reliable performance for traders.

You will join a growing team of four in Sydney within the APAC Data Team and collaborate with counterparts in Chicago, Mumbai and Amsterdam.

The role emphasizes systems automation, CI/CD, and adoption of modern tools such as Kafka, Kubernetes, Dremio and Airflow to deliver fast, reliable insights.

Qualifications

• 2+ years of experience in managing large-scale multi-petabyte data infrastructure.
• Knowledge of Linux system administration and internals, with proven ability to troubleshoot issues in Linux environments.
• Experience in at least one of the following technologies: Kafka, HDFS, Kubernetes.
• Working knowledge of Docker, Kubernetes, and Helm.
• Experience with data access technologies such as Dremio and Presto.
• Familiarity with workflow orchestration tools like Airflow and Prefect.
• Exposure to cloud platforms such as AWS, GCP, or Azure.

Responsabilités

• Manage and monitor distributed systems, storage infrastructure, and data processing platforms, including Kafka, HDFS, Dremio, and in-house data pipelines.
• Drive heavy focus on systems automation and CI/CD to enable rapid deployment of hardware and software solutions.
• Collaborate closely with systems and network engineers, traders, and developers to support and troubleshoot their queries.
• Stay up to date with the latest technology trends in the industry; propose, evaluate, and implement innovative solutions.

Connaissances

Linux administration

CI/CD automation

Distributed systems

Troubleshooting

Cross-team collaboration

Outils

Kafka

HDFS

Dremio

Docker

Kubernetes

Helm

Airflow

Prefect

AWS

GCP

Azure

Description du poste

IMC in Sydney seeks a Data Engineer to own large-scale data collection, storage, access and query platforms, ensuring safe operation and reliable performance for traders.

You will join a growing team of four in Sydney within the APAC Data Team and collaborate with counterparts in Chicago, Mumbai and Amsterdam.

The role emphasizes systems automation, CI/CD, and adoption of modern tools such as Kafka, Kubernetes, Dremio and Airflow to deliver fast, reliable insights.