Backend Developer

Purple Sphere; California (MO); Hybrid; USD 120,000 - 170,000; Full time

Job summary

Purple Sphere is seeking a backend engineer focused on real-time streaming systems to design and operate scalable data pipelines.

You will work with Kafka, Flink, MQTT, and Kubernetes to build robust, low-latency services, mentor juniors, and collaborate with cross-functional teams on production-grade streaming architectures.

Experience with time-series databases, NoSQL stores, and cloud platforms is preferred.

Qualifications

• 3+ years in backend development with 2+ years in real-time streaming systems.
• Production systems handling >100K messages/second.
• Strong data structures and algorithms foundation for complex problems.
• Proficiency in Python and Go with async programming knowledge.
• Expertise with Kafka (incl. Streams), Flink, and MQTT.
• Docker and Kubernetes for distributed app deployment.
• Experience with time-series DBs and NoSQL stores.

Responsibilities

• Design and implement distributed, fault-tolerant streaming pipelines using Kafka, Flink, and MQTT.
• Build event-driven microservices for real-time data processing and analytics.
• Optimise streaming applications for performance, throughput, and latency.
• Deploy and manage containerised applications on Kubernetes clusters.
• Implement monitoring, alerting, and auto-scaling strategies for streaming workloads.
• Troubleshoot production issues and ensure system reliability.
• Collaborate with cross-functional teams and mentor junior developers.

Skills

Backend development

Real-time streaming

Python

Golang

Data structures

Algorithms

Distributed systems

Async programming

Tools

Kafka

Kafka Streams

Apache Flink

MQTT

Docker

Kubernetes

Cassandra

MongoDB

Job description

Responsibilities:

• Design and implement distributed, fault-tolerant streaming pipelines using Kafka, Apache Flink, and MQTT.
• Build event-driven microservices for real-time data processing and analytics.
• Optimise streaming applications for performance, throughput, and latency.
• Deploy and manage containerised applications on Kubernetes clusters.
• Implement monitoring, alerting, and auto-scaling strategies for streaming workloads.
• Troubleshoot production issues and ensure system reliability.
• Collaborate with cross-functional teams and mentor junior developers.

Requirements:

• 3+ years in backend development with 2+ years in real-time streaming systems.
• Proven track record of building production systems handling and gt 100K messages/second.
• Data Structures and Algorithms: Extremely strong foundation in data structures and algorithms with the ability to design optimal solutions for complex problems.
• Programming: Strong proficiency in Python and Golang, with deep understanding of concurrent and async programming.
• Streaming Systems: Expert-level proficiency with Apache Kafka (including Kafka Streams), hands-on experience with Apache Flink, and working knowledge of the MQTT protocol.
• Containerization: Proficiency with Docker and Kubernetes, including deployment and orchestration of distributed applications.
• Distributed Systems: Solid understanding of distributed systems concepts, event-driven architecture, and streaming design patterns.
• Databases: Experience with time-series databases and NoSQL stores (Cassandra, MongoDB).

Preferred Skillset:

• Experience with ELK Stack (Elasticsearch, Logstash, Kibana) for log aggregation and analytics.
• Cloud platform experience (AWS, GCP, or Azure).
• Knowledge of Apache Beam or similar unified processing frameworks.
• Experience with ML/AI integration in streaming pipelines.
• Open-source contributions to streaming projects.