ML Operations Engineer (MSA)

A better internet, where privacy is the default, is possible. Building this better internet might seem daunting or even unthinkable, but at Proton, this is what we do every day.

Proton was founded in 2014 by a group of scientists who met at the European Organization for Nuclear Research (CERN). Our first product, Proton Mail, is now the world’s largest encrypted email service. Subsequent Proton products, such as Proton VPN, Proton Calendar, Proton Drive, and Proton Pass give our users full control over how and with whom their data is shared.

Today, Proton makes privacy universally accessible. Journalists from outlets like The Guardian and The New York Times, some of the world’s largest organizations, and people in more than 180 countries have signed up for over 100 million accounts. Our diverse and dynamic team is made up of more than 500 members representing over 40 different nationalities. While we are based in Geneva, Switzerland, we have offices in Zurich, Prague, London, Vilnius, Skopje, Taipei, Paris, Barcelona and many more employees working remotely around the world.

Join one of Europe’s fastest-growing companies to help us solve challenging problems and build new products that will reach hundreds of millions of people. We want to create more than just one of the world’s most impactful tech companies; we want to create a new internet that serves the interests of all people. We need you, your voice, your ideas, and your ambition to make it happen.

Purpose of the role:

MSA is a highly multi-disciplinary group started in 2019 to tackle difficult mail delivery, spam, abuse, and account security problems that impact the Proton ecosystem. We built sophisticated systems from scratch that combine human intelligence and machine learning to make tens of millions of realtime or asynchronous decisions each day. As of late 2023, our areas of focus are 1) mail delivery and spam, 2) abuse and account security, 3) site reliability of Proton, and 4) a new product involving blockchains. Over the past few years, our custom systems have reduced spam filter misclassifications by over 60%, blocked millions of abusive bulk signups, protected hundreds of thousands of users from account compromise by attackers, and mitigated hundreds of DDoS attacks. In order to move fast with limited resources, we are organized into autonomous teams responsible for the whole system, with expertise across infrastructure, data storage, backend, web and mobile apps, machine learning, security and operational excellence. In 3 years, we grew from 2 engineers to over 30 engineers and analysts, spread across 3 continents for 24/7 coverage. We are looking for humble, mission-driven, systems-thinking people who want to make a big impact in a startup environment.

What you will do:

As an ML Operations Engineer at Proton, you will be at the forefront of maintaining the operational excellence of our machine learning and anti-abuse systems. These critical services must run reliably 24/7 and scale with increasing traffic and complexity. Your responsibilities will include the research, development, stress testing, and rollout of new services and infrastructure. You will collaborate with Software and ML Engineers to ensure that systems have comprehensive logging, monitoring, alerting, and on-call capabilities. Your expertise in debugging and problem-solving will be essential in addressing issues that arise in production environments.

Key responsibilities:

1. Ensure the operational excellence of machine learning and anti-abuse systems running 24/7.
2. Research, develop, stress test, and roll out new services and infrastructure.
3. Collaborate with engineers to implement logging, monitoring, alerting, and on-call capabilities.
4. Debug and resolve production issues efficiently.
5. Manage and monitor production systems involving in-house developed solutions.
6. Deploy, run, and maintain machine learning systems in production environments.
7. Utilize Docker for containerization and efficient deployment.
8. Implement and manage Continuous Integration/Continuous Deployment (CI/CD) pipelines.

Job requirements:

1. Strong expertise in infrastructure automation tools (Puppet, Terraform, OpenNebula, Kubernetes).
2. Proficient in scripting languages (Bash, Python).
3. Experience in managing and scaling data storage solutions, particularly MySQL, Kafka, and ElasticSearch.
4. Familiarity with Dask and Airflow for data processing and workflow orchestration.
5. Solid experience with CI/CD pipelines for automated deployments.
6. Hands-on experience with Docker for containerization.
7. Ability to operate in a fast-paced environment with a global team and participate in on-call rotations.
8. Excellent communication skills in English, both written and spoken.

Desirable:

1. Past experience in running ML systems in a production environment.
2. Experience in establishing and managing ML platforms, including feature stores and model registries/hubs.

Why should you join Proton?

1. Be part of a movement -Proton is not just a product or service but a community-driven movement united by a shared vision of online freedom. Our services are open source, audited, and supported by community contributions. We give back to our community by maintaining core encryption libraries and by supporting other organizations furthering the same goals as us. Proton is free, open source, neutral, independent, and community first, while remaining financially sustainable.
2. Work with smart and dedicated people -Our team is diverse, collaborative, and tight-knit with people coming from all walks of life, including many of the world’s top academic institutions and organizations, such as MIT, Harvard, Stanford, Caltech, Cambridge, and ETH.
3. Join a strong brand -Our encrypted email service - ProtonMail - has grown to be a staple of online security and privacy. Proton has been featured in multiple popular television and film productions, such as Mr. Robot, Knives Out, Sounds of Metal, and more.
4. Grow with us -We’re one of Europe’s fastest-growing startups, doubling in size every year. Our growth gives you limitless career and educational opportunities as well as the opportunity to work side-by-side with many world-leading experts in their fields.
5. Have your voice heard -We value your opinion and encourage you to speak up and share your ideas and thoughts. At Proton, no problem is someone else’s problem. We collectively strive to do the right thing and be the undisputed best in the world at everything we do.
6. Benefits –these vary by location and type of contract but expect support on your vacation, parental leave, refreshment if working from the office, learning and development opportunities, equity for shared success, flexible working hours and remote work, company events and team building activities.

Proton does not accept unsolicited resumes from any sources other than directly from a candidate. Proton will not pay a fee for any placement resulting from the receipt of an unsolicited offer, even in a situation when the relevant candidate is employed by Proton.