Remote Real-Time Data Engineer - Kafka Platform

United States Digital Space LLC; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking a Real-Time Data Engineer to architect, deploy, and operate large-scale Apache Kafka environments across on-prem and cloud. You will own the platform end-to-end, including sizing, security, automation, and developer enablement.

The ideal candidate has deep Kafka internals knowledge, strong DevOps/SRE practices, and the ability to mentor others while collaborating with product, design, engineering, and operations to deliver reliable streaming solutions.

Qualifications

• Five or more years of experience operating Apache Kafka or Confluent Platform in production.
• Hands-on knowledge of Kafka internals (partitions, replication, ISRs, consumer groups).
• Experience with Kafka security (SASL, mTLS, ACLs, RBAC).
• Experience with Kafka Connect, Schema Registry, and Kafka Streams or ksqlDB.
• Experience with HA/DR strategies for Kafka.

Responsibilities

• Architect, deploy, and operate large-scale Kafka/Confluent Platform clusters across on-prem and cloud environments.
• Design partitioning, replication, and topic strategies balancing throughput and durability.
• Implement security on Kafka clusters using SASL, mTLS, ACLs, RBAC, and IdP integration.
• Build CI/CD pipelines for Kafka configurations using GitOps patterns.
• Maintain observability with Prometheus, Grafana, Datadog, or Confluent Control Center.
• Mentor junior engineers and contribute to runbooks and design decisions.

Skills

Kafka internals

Kafka security

Scripting (Python/Bash/Go)

Infrastructure as Code

Observability tooling

Troubleshooting & communication

Education

Bachelor’s degree in Computer Science/Engineering

Tools

Terraform

Ansible

Job description

Bright Vision Technologies is seeking a Real-Time Data Engineer to architect, deploy, and operate large-scale Apache Kafka environments across on-prem and cloud. You will own the platform end-to-end, including sizing, security, automation, and developer enablement.

The ideal candidate has deep Kafka internals knowledge, strong DevOps/SRE practices, and the ability to mentor others while collaborating with product, design, engineering, and operations to deliver reliable streaming solutions.