Lead Kafka Platform Engineer - Remote

Socket.dev; Cranberry Township (Butler County); On-site; USD 100,000 - 150,000; Full time

Job summary

Bright Vision Technologies is seeking a seasoned Kafka Engineer to architect, deploy, and operate large-scale Apache Kafka and Confluent environments supporting mission-critical workloads. This role spans cluster sizing, security, automation, and developer enablement across on-prem and cloud.

You'll collaborate with product, design, engineering, operations, and business stakeholders to translate requirements into robust streaming solutions, mentor engineers, and contribute to runbooks and

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related technical discipline.
• Five or more years of experience operating Apache Kafka or Confluent Platform in production.
• Deep, hands-on knowledge of Kafka internals (partitions, replication, ISRs, consumer groups).
• Strong experience with Kafka security (SASL, mTLS, ACLs, RBAC).
• Hands-on experience with Kafka Connect, Schema Registry, and either Kafka Streams or ksqlDB.
• Experience with HA/DR strategies for Kafka.
• Strong scripting skills in Python, Bash, or Go.
• Hands-on experience with infrastructure-as-code (Terraform, Ansible).
• Working knowledge of observability tooling for Kafka.
• Excellent troubleshooting, communication, and documentation skills.

Responsibilities

• Architect, deploy, and operate large-scale Apache Kafka or Confluent Platform clusters across on-prem and cloud environments.
• Design partitioning, replication, and topic strategies that balance throughput, durability, and operational simplicity.
• Implement strong security on Kafka clusters using SASL, mTLS, ACLs, RBAC, and integration with corporate IdPs.
• Operate Schema Registry, Kafka Connect, KSQL/ksqlDB, and Kafka Streams in production.
• Build and operate Kafka Connect pipelines integrating sources and sinks across enterprise systems.
• Design HA/DR strategies for Kafka, including MirrorMaker 2, Cluster Linking, and multi-region active-active patterns.
• Build CI/CD pipelines for Kafka topic, ACL, and connector configurations using GitOps patterns.
• Implement comprehensive observability using Prometheus, Grafana, Datadog, or Confluent Control Center.
• Drive Kafka cost and capacity optimization through right-sizing and storage tiering.
• Onboard application teams to Kafka with clear patterns, templates, and best practices.
• Lead incident response and post-incident reviews for streaming workloads, applying disciplined engineering practices and partnering closely with stakeholders to ensure outcomes are durable, well-documented, and aligned with broader team and platform standards.
• Mentor and coach junior and mid-level engineers through code review, design review, pair programming, and structured knowledge sharing, helping the broader team grow in technical maturity and confidence over time.
• Maintain comprehensive, current technical documentation - including architecture diagrams, design decisions, configuration references, runbooks, and operational procedures - so that the system remains supportable, auditable, and easy to onboard new engineers over time.
• Continuously evaluate emerging streaming technologies (Pulsar, Redpanda, AWS MSK, Azure Event Hubs).

Skills

Kafka internals

Kafka security

Kafka Connect

Schema Registry

KSQL/ksqlDB

Scripting: Python/Bash/Go

Terraform/Ansible

Observability tooling

Communication & documentation

Kubernetes

Education

Bachelor’s degree in Computer Science/Engineering

Tools

Terraform

Ansible

Kafka Connect

Schema Registry

KSQL/ksqlDB

Kubernetes

Python

Bash

Go

Job description

Bright Vision Technologies is seeking a seasoned Kafka Engineer to architect, deploy, and operate large-scale Apache Kafka and Confluent environments supporting mission-critical workloads. This role spans cluster sizing, security, automation, and developer enablement across on-prem and cloud.

You'll collaborate with product, design, engineering, operations, and business stakeholders to translate requirements into robust streaming solutions, mentor engineers, and contribute to runbooks and