Data engineer

Macquarie Group; Greater London; Hybrid; GBP 90,000 - 120,000; Full time

Benefits offered by this job

Wellbeing leave

Parental leave (26 weeks)

Fertility leave

Volunteer leave and matching

Salary sacrifice options

Medical and life insurance

Employee Assistance Program

Learning & development

Home office equipment reimbursement

Flexible working

Job summary

Macquarie Group ищет специалиста по данным в команду Data Services. Ваша задача — проектировать и строить масштабную облачную платформу данных, управлять Data Lake и обеспечивать безопасные, масштабируемые окружения.

Будете работать с AWS S3, Airflow, Trino, Parquet и Redshift, оптимизируя запросы на триллионные объёмы. Проживание в гибридном формате и возможность влиять на стратегию компании.

Qualifications

• Сильный бэкграунд в Data, DevOps и инженерии, решение задач по данным и автоматизации.
• Опыт создания CI/CD пайплайнов в Data space и облачной инфраструктуре.
• Хорошее понимание современных Data Platforms, баз данных, Data Lakes, Data Warehouses и Query Engines.
• Опыт программирования на Python или работа с любыми облачными технологиями (AWS предпочтительно).
• Хорошее понимание Hosting Platform Linux/Unix.
• Будет плюсом опыт работы с EKS и контейнерами.

Responsibilities

• Проектирование, сборка и оптимизация облачной платформы данных масштаба петабайт на базе современных технологий.
• Управление и расширение Data Lake и безопасной, масштабируемой средой для наших Data Platform.
• Использование облачных технологий и инструментов AWS: S3, EKS, Glue, Airflow, Trino, Parquet, с возможной подготовкой Iceberg.
• Решение задач по высоким нагрузкам и чтению крупных объёмов данных (600+ миллиардов строк в Redshift).
• Прямая работа в составе команды и вклад в стратегию компании, работа над разнообразными проектами.

Skills

Data engineering

DevOps

Python

SQL

AWS

Linux/Unix

Container experience

Tools

Airflow

Trino

Parquet

Redshift

Glue

AWS S3

EKS

Job description

Описание:

At Macquarie, our Data Services Team drives innovation by delivering a diverse suite of data products that empower data-driven decision-making and enhance operational efficiency. With a strong focus on engineering, data science, and analytics, the team delivers innovative solutions that empower business users primarily trading teams to tackle complex challenges and unlock new opportunities.

Задачи:

designing, building, and optimising a cutting-edge, petabyte-scale data platform powered by modern cloud technologies. manage and enhance a petabyte-scale Data Lake create secure, efficient, and scalable environments for our data platforms. You'll leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You'll tackle high-performance data workloads, ensuring seamless execution of massive queries, including 600+ billion-row queries in Redshift, by designing and maintaining robust, scalable infrastructure. Our flat structure means that you will directly contribute to our strategy while taking ownership of a diverse range of projects utilising the latest technologies.

Требования:

A dynamic individual with a strong Data, DevOps and Engineering background who loves to solve data problems and build automation and CI/CD Pipelines in the data space and cloud infrastructure. Good understanding of modern Data Platforms, Databases, Data Lakes, Data Warehouses and Query Engines, SQL/DDLs Experience with Python coding/testing or any Cloud-based technology (AWS preferred) Good understanding of Hosting Platform Linux/Unix Будет плюсом: EKS and Container experience is a plus

Условия:

• 1 wellbeing leave day per year and a minimum of 25 days of annual leave.
• 26 weeks' paid parental leave for primary caregivers along with 12 days of paid transition leave upon return to work and 6 weeks' paid leave for secondary caregivers
• Paid fertility leave for those undergoing or supporting fertility treatment
• 2 days of paid volunteer leave and donation matching
• Access to a wide range of salary sacrificing options
• Benefits and initiatives to support your physical, mental and financial wellbeing including, comprehensive medical and life insurance cover
• Access to our Employee Assistance Program, a robust behavioural health network with counselling and coaching services
• Access to a wide range of learning and development opportunities, including reimbursement for professional membership or subscription
• Access to company funded emergency and backup dependent care services
• Recognition and service awards
• Hybrid and flexible working arrangements, dependent on role
• Reimbursement for work from home equipment