Platform Staff Engineer

DISCO; Greater London; On-site; GBP 150,000 - 190,000; Full time

Benefits offered by this job

Start-up culture

Open PTO

Growth opportunities

Medical, dental, and vision, 401(k)

Job summary

DISCO is seeking a Staff Software Engineer (Platform) in London to lead large-scale projects and shape the group’s technical direction. You will mentor senior engineers and influence decisions across multiple teams, focusing on system design and the evolution of the distributed data platform.

You will own performance, scalability, and reliability while collaborating with product managers, designers, and executives to align architectural plans with business goals.

Qualifications

• 10+ years in backend engineering with big data platforms.
• Proven experience with JVM stacks, Kotlin, and gRPC.
• Strong design, scalability, and performance focus; cloud familiarity.

Responsibilities

• Lead technical direction for large projects and mentor engineers.
• Design and maintain high-performance data platforms.
• Collaborate with product, design, and other engineers to align architecture with business needs.

Skills

Backend engineering

Distributed systems

Big data platforms

Kubernetes

Docker

Java/Kotlin

Python

REST APIs

gRPC

CI/CD

Observability

Cloud platforms (AWS/Azure/GCP)

Tools

Jenkins

GitLab CI

CircleCI

Envoy

AWS App Mesh

Job description

• The Staff Software Engineer (Platform) leads large-scale projects and is a key contributor to their group’s success
• They are involved in long-term strategic planning, mentor senior engineers, and are influential in technical decision-making across multiple teams or a department
• They focus on implementation and system design, as well as maintaining and evolving DISCO’s distributed data platform
• Technical Leadership and Mentorship: Provides technical leadership in project teams, guiding engineers, and contributing to technical decision-making. Sets technical direction for large projects or teams, driving innovation, and solving organization-wide problems
• Performance Optimization: Focuses on designing systems for high performance, scalability, and reliability
• Ownership: Proactively identifies areas of products and systems that currently have problems or will develop problems as well as makes plans to address those problems
• Stakeholder Engagement: Works with stakeholders, including product managers, designers, developers, and executives, to align architectural designs with business needs and customer expectations
• Software Development: Designs, develops, and maintains sophisticated, scalable, high-performance data software solutions, ensuring they are efficient, scalable, and meet business
• Equirements with architectural excellence for our big data platform at petabyte scale. Responsible for the boundary identification, system design and grooming for one or more sub-systems
• Code Quality and Optimization: Upholds and enhances code quality through rigorous review processes, optimization techniques, and adherence to best practices
• Technical Problem Solving: Addresses intricate technical issues through innovative problem-solving strategies
• Collaboration and Communication: Works effectively within the engineering team and cross-functional teams, collaborating with other engineers, product managers, and stakeholders, contributing to project planning and execution
• Continuous Learning and Adaptation: Stays updated with the latest technology trends and advancements, continually enhancing skills and knowledge, bringing innovative ideas to the team
• Technical Documentation: Creates and maintains comprehensive technical documentation, ensuring clarity and accessibility of information
• Data Platform Development: Builds and maintains scalable and robust data platforms to support analytics, machine learning, and business intelligence
• Data Integration and ETL: Develops and optimizes ETL (Extract, Transform, Load) processes and data integration pipelines. Ensures efficient data flow across various sources and systems
• Advanced File Processing: Develops extensible file processing pipelines that offer flexible, self-service artifact generation. Researches and evaluates file processing software based on fidelity, reliability, and performance criteria. Models file processing outcomes within an observability framework to provide insights for engineering teams and business leaders. Develops engineering systems that facilitate rapid and automatic evaluation of file processors
• Data Management: Crafts and orchestrates scaleout data pipelines that optimize serverless and containerized compute to balance cost, latency, and duration for big-data solutions. Contributes to algorithmically intensive data engines operating on streaming, large, or multi-tenant datasets. Troubleshoots systems, identify root causes, and rectify bugs

Benefits

• Start-up culture
• Open PTO
• Growth opportunities
• Medical, dental, and vision, 401(k)

Experience leveraging observability systems (logging, metrics, traces) to monitor and debug production subsystems and generate business-level metricsDemonstrated expertise in designing, implementing, and maintaining (through operational observability) high-availability, high-performance, distributed data processing systemsExperience with Large-Scale Systems: Involvement in developing, monitoring, and maintaining large-scale, high-availability software systemsExperience with designing, modifying, and operating multi-tenant systemsFamiliarity with Agile development methodologies and practices10+ years of relevant experience in backend engineering, with a substantial focus on big data platformsExperience with gRPC and Protocol Buffers for efficient, language-agnostic service-to-service communicationExperience selecting and extending 3rd party components (commercial or open source) that provide operational leverageExperience using feature or release toggles as a code branching strategyExperience designing and developing from a Security PerspectiveDeep expertise with both relational and non-relational data stores: Elasticsearch, Cassandra, MySQL, and PostgreSQLDemonstrated experience in leading or playing key roles in significant software projects, showing an ability to work effectively both independently and as part of a teamExperience with CI/CD pipelines and tools (like Jenkins, GitLab CI, or CircleCI) for automated testing and deployment of microservicesExperience with tools such as Envoy or AWS App MeshProven ability to deliver well-crafted, tested, and maintainable code solutions to complex technical challengesExperience with a variety of software stacks including, but not limited to, JVM-focused tech stacks, particularly server-side Kotlin and gRPCExperience with algorithmically intensive data engines operating on streaming, large, or multi-tenant datasetsExperience delivering data platform software solutions (including designing, implementing, and operating) in the cloud7+ years of professional experience with each of the following: Service-Oriented Architecture, Docker-based services, microservices, API, and Data Model Design or ImplementationFamiliarity with server management, network configurations, and an understanding of cloud services (like AWS, Azure, or Google Cloud)Ability to build backend systems that can support future UI/UX initiatives and performance requirementsKnowledge of system architecture, design patterns, and best practices in software developmentSkilled in containerization technologies such as Docker and orchestration tools like Kubernetes. Experience in creating, deploying, and managing containersSkilled in programming languages and frameworks commonly used in SaaS development. In particular, Java/Kotlin, Python, C#, as well as a deep understanding of software engineering principlesSkilled at designing and operating software in a Cloud Provider, such as AWS, Azure, or GCPKnowledge of both relational and non-relational data stores, Elasticsearch, Cassandra, MySQL, PostgreSQLSkilled in implementing RESTful APIs for outward-facing services and using gRPC for efficient internal service-to-service communicationKnowledge of microservices architecture principles, including decentralized data management, service discovery, and bounded contexts. Skilled in microservices technology stack, ability to optimize performance, and ensure security and scalability across the systemSkilled at analytical abilities and problem-solving, capable of addressing complex technical challengesKnowledge of async message or event based dataflow architectures (Kafka, etc)Knowledge of design, implementation, and operation of data-intensive, distributed systemsKnowledge of various stages of the SDLC, including design, development, testing, and deployment