Data Platform Engineer

TN United Kingdom; London; On-site; GBP 45,000 - 85,000; Full time

Job description

Social network you want to login/join with:

We are the leading independent provider of harmonised Level 3 historical data and analytics to the world’s most sophisticated capital market participants. BMLL offers banks, brokers, asset managers, hedge funds and global exchange groups immediate and flexible access to the most granular Level 3, T+1 order book data and advanced analytics, enabling them to accelerate research, optimise trading strategies and generate alpha at unparalleled speed and scale.

Our culture is inclusive and highly collaborative, with a flat management structure that empowers our employees to get involved in decision making as we continue to grow and scale. We give all our employees share options so they participate in the growth and development of the business.

We offer a combination of remote and office (London based) working, weekly team lunches and plenty of office snacks!

About the Role

At BMLL, we process terabytes of historical market data every day, for which we have a powerful data processing platform built on AWS to provide best-in-class capacity, scalability and reliability.

We are looking for a Data Platform Engineer to join BMLL's Core Engineering team, where you'll architect the core platform in which BMLL's development teams execute highly complex data pipelines. You'll design and build solutions that scale compute to millions of concurrent job executions, optimised to meet performance, efficiency and cost-effectiveness requirements, while ensuring high availability.

Data Platform Engineers bridge the gap between software and infrastructure, and are essential to the success of BMLL's technology strategy.

Responsibilities

• Design and build solutions to scale AWS compute resources to meet application performance requirements.
• Ensure 24/7 system reliability by implementing company and industry best practices in replication, redundancy and monitoring.
• Implement workflow management software, to automate operational tasks and optimise the utilisation of infrastructure and applications.
• Design and implement CI/CD workflows to maintain software quality via continuous and automated deployment and testing.
• Work with development and operations teams to design solutions to complex problems, involving large data pipelines that process terabytes of historical market data, in the most efficient and cost-effective manner.
• Regularly review and assess new tools that become available in the industry and assess how they could be integrated into the platform to continuously improve.

Requirements

• Industry experience with cloud computing tools and services in complex systems, preferably in AWS.
• Industry experience with software development lifecycle processes and tools.
• Experience working in a Linux environment.
• Experience with Docker.
• Experience with SQL and relational databases.
• Avid learner, problem solver and detail-orientated.
• Excellent teamwork and the ability to communicate and work in multidisciplinary teams in a collaborative manner.
• Computer science or other STEM degree.
• At least two years of industry experience.
• Familiarity with distributed systems concepts and tools, such as Spark, Ray, RabbitMQ, Kafka, AWS Batch.
• Familiarity with DevOps practices and tools, such as Terraform.
• Familiarity with job execution and orchestration tools, such as Celery and Airflow.

We are an inclusive employer and welcome applicants from all backgrounds. We pride ourselves on our commitment to Equality and Diversity. We are committed to removing barriers throughout our hiring process. If you have any special requirements or require reasonable adjustments to help you access career opportunities at BMLL, please do let us know.