Data Platform SRE: Scale & Automate Data Pipelines

IMC Trading; Sydney; On-site; AUD 120,000 - 170,000; Full time

Job summary

IMC Trading in Sydney seeks a Data Platform Engineer to manage large-scale data infrastructure and ensure reliable operation of data platforms. You will drive automation, collaborate with traders and engineers, and contribute to scalable data solutions across teams in APAC and globally.

The role involves working with Kafka, HDFS, Kubernetes, and cloud platforms, fostering a high-performance data culture within a global trading environment.

Qualifications

• 2+ years of experience managing large-scale multi-petabyte data infrastructure.
• Strong Linux system administration and troubleshooting skills.
• Experience with Kafka, HDFS or Kubernetes.
• Working knowledge of Docker, Kubernetes, and Helm.
• Experience with data access technologies such as Dremio and Presto.
• Familiarity with workflow tools like Airflow and Prefect.
• Exposure to AWS, GCP, or Azure.

Responsibilities

• Manage and monitor distributed systems, storage, and data processing platforms.
• Drive automation and CI/CD for rapid deployment of hardware and software.
• Collaborate with systems, network engineers, traders, and developers to support queries.
• Stay updated with technology trends and implement innovative solutions.

Skills

Data infrastructure

Linux administration

Kafka

HDFS

Kubernetes

Docker

Airflow

Prefect

Cloud platforms

CI/CD

Education

Tools

Dremio

Presto

Helm

Docker

Kubernetes

Job description

IMC Trading in Sydney seeks a Data Platform Engineer to manage large-scale data infrastructure and ensure reliable operation of data platforms. You will drive automation, collaborate with traders and engineers, and contribute to scalable data solutions across teams in APAC and globally.

The role involves working with Kafka, HDFS, Kubernetes, and cloud platforms, fostering a high-performance data culture within a global trading environment.