Senior Streaming Data Engineer — Remote Kafka Platform

United States Digital Space LLC; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking an experienced Streaming Data Engineer to architect, deploy, and operate large-scale Apache Kafka and Confluent platform environments supporting mission-critical workloads. You will own the Kafka platform end-to-end, including sizing, security, automation, observability, and developer enablement.

The ideal candidate has deep Kafka internals knowledge, strong DevOps/SRE practices, and will partner with product and engineering teams to deliver a reliable,

Qualifications

• Five+ years of production experience with Apache Kafka or Confluent Platform.
• Deep knowledge of Kafka partitions, replication, ISRs, and consumer groups.
• Experience with Kafka security (SASL, mTLS, ACLs, RBAC).
• Proficiency in Python, Bash, or Go and IaC tooling.
• Strong communication and documentation skills.

Responsibilities

• Architect, deploy, and operate large-scale Kafka clusters across on-prem and cloud.
• Design partitioning, replication, and topic strategies for throughput and durability.
• Implement security on Kafka clusters using SASL, mTLS, ACLs, and RBAC.
• Maintain Schema Registry, Kafka Connect, and Kafka Streams in production.
• Build CI/CD pipelines for Kafka configurations using GitOps.
• Develop observability using Prometheus, Grafana, Datadog, or Confluent Control Center.
• Lead incident response and post-incident reviews for streaming workloads.
• Mentor junior engineers and maintain runbooks and docs.

Skills

Kafka administration

Kafka internals

Python scripting

Bash scripting

Go scripting

IaC (Terraform/Ansible)

Observability tooling

Troubleshooting

Documentation

Tools

Kafka Connect

Schema Registry

KSQL/ksqlDB

Kafka Streams

Prometheus

Grafana

Datadog

Confluent Platform

Terraform

Ansible

Job description

Bright Vision Technologies is seeking an experienced Streaming Data Engineer to architect, deploy, and operate large-scale Apache Kafka and Confluent platform environments supporting mission-critical workloads. You will own the Kafka platform end-to-end, including sizing, security, automation, observability, and developer enablement.

The ideal candidate has deep Kafka internals knowledge, strong DevOps/SRE practices, and will partner with product and engineering teams to deliver a reliable,