Real-Time Kafka Engineer for Scalable Data Pipelines

Staffing Spot, Inc.; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Staffing Spot, Inc. is seeking a Kafka Engineer to design, develop, and operate scalable event-streaming pipelines using Apache Kafka and related technologies in a cloud-native environment.

You will collaborate with data engineers to implement high-availability architectures, monitor performance with Prometheus and Grafana, and automate deployments using CI/CD within a Kubernetes/Docker stack to ensure robust, scalable data platforms.

Qualifications

• Experience with Apache Kafka, including producers and consumers.
• Strong knowledge of Kafka topics, partitions, offsets and replication.
• Experience with Kafka Connect and Kafka Streams.
• Hands-on experience with Docker and Kubernetes in production.
• Familiarity with monitoring: Prometheus and Grafana.

Responsibilities

• Design and implement scalable Kafka-based event-streaming solutions.
• Develop and maintain Kafka producers and consumers.
• Create and manage Kafka topics, partitions, and consumer groups.
• Configure and monitor Kafka brokers and clusters.
• Implement high availability, fault tolerance, and disaster recovery.

Skills

Kafka expertise

Real-time pipelines

CI/CD scripting

Troubleshooting

Team collaboration

Tools

Kafka Connect

Kafka Streams

Docker

Kubernetes

Prometheus

Grafana

Job description

Staffing Spot, Inc. is seeking a Kafka Engineer to design, develop, and operate scalable event-streaming pipelines using Apache Kafka and related technologies in a cloud-native environment.

You will collaborate with data engineers to implement high-availability architectures, monitor performance with Prometheus and Grafana, and automate deployments using CI/CD within a Kubernetes/Docker stack to ensure robust, scalable data platforms.