Kafka Platform Engineer

Bright Vision Technologies; Alpharetta (GA); Remote; USD 130,000 - 180,000; Full time

Job summary

Bright Vision Technologies is seeking a highly experienced Kafka Platform Engineer with 10+ years in distributed systems, enterprise streaming, and platform engineering to architect, deploy, and manage enterprise-scale streaming platforms powering real-time data across the organization.

This role emphasizes deep Kafka internals, platform automation, security, observability, and mentoring engineering teams, with a focus on enterprise event-streaming standards and cloud-native infrastructure.

Qualifications

• 10+ years of professional experience in distributed systems or platform engineering.
• 5+ years hands-on Apache Kafka or Confluent Platform experience.
• Expert knowledge of Kafka internals (partitions, replication, brokers, ISR, consumer groups, offset management).
• Experience implementing Kafka security (SASL, SSL/mTLS, ACLs, RBAC).
• Hands-on with Kafka Connect, Schema Registry, Kafka Streams, ksqlDB, and real-time streaming architectures.
• HA/DR, multi-cluster replication, cross-region failover experience.
• Scripting with Python/Bash/Go; IaC with Terraform/Ansible.
• Observability with Prometheus, Grafana, OpenTelemetry, ELK; distributed tracing.

Responsibilities

• Architect, deploy, and manage highly available Apache Kafka and Confluent Platform environments for enterprise-scale applications.
• Design Kafka cluster topology, broker configuration, partitioning, replication, capacity planning, and performance optimization.
• Implement security using SASL, SSL/mTLS, ACLs, RBAC; ensure encryption and authentication.
• Design and manage Kafka Connect, Schema Registry, Kafka Streams, ksqlDB pipelines for real-time data integration.
• Develop Infrastructure as Code using Terraform or Ansible to provision Kafka infrastructure.
• Design and implement HA, DR, backup, failover, and cross-region replication strategies.
• Build monitoring and observability using Prometheus, Grafana, OpenTelemetry, ELK, and Confluent tools.
• Optimize cluster performance, troubleshoot production issues, perform root-cause analysis, and drive platform improvements.
• Support Kafka deployments on Kubernetes using Strimzi, Confluent Operator, or cloud services (AWS MSK, Azure Event Hubs Kafka API, Confluent Cloud).
• Collaborate with developers, data engineers, DevOps, SRE, and enterprise architects to set standards and governance.
• Conduct architecture reviews and mentor engineers on Kafka development and operations.

Skills

Distributed systems

Platform engineering

Apache Kafka

Confluent Platform

Kafka security

Kubernetes

Terraform

Ansible

Prometheus

Grafana

OpenTelemetry

ELK

Education

Bachelor's or Master's degree in CS/Engineering

Tools

Terraform

Ansible

Kubernetes

Confluent Platform

Job description

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.

Job Title: Kafka Platform Engineer

Location: 100% Remote (U.S.)

Position Type: Full-time, Direct W2

Salary Range: $130,000-$180,000 Annually (based on experience)

Experience Required: 10+ Years

Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.

Job Summary

Bright Vision Technologies is seeking a highly experienced Kafka Platform Engineer with 10+ years of experience in distributed systems, event streaming, and platform engineering, including extensive expertise with Apache Kafka and the Confluent Platform. The ideal candidate will architect, deploy, and manage enterprise-scale streaming platforms that power mission-critical, real-time data processing across the organization. This role requires deep technical expertise in Kafka internals, platform automation, security, observability, and cloud-native infrastructure, along with the ability to mentor engineering teams and define enterprise event-streaming standards.

Key Responsibilities

• Architect, deploy, and manage highly available Apache Kafka and Confluent Platform environments supporting enterprise-scale event-driven applications.
• Design Kafka cluster topology, broker configuration, partitioning strategies, replication, capacity planning, and performance optimization for high-throughput workloads.
• Implement enterprise-grade security using SASL, SSL/mTLS, ACLs, RBAC, encryption, and authentication mechanisms.
• Design and manage Kafka Connect, Schema Registry, Kafka Streams, ksqlDB, and event streaming pipelines for real-time data integration.
• Develop and maintain Infrastructure as Code using Terraform, Ansible, or similar automation frameworks to provision and manage Kafka infrastructure.
• Design and implement High Availability (HA), Disaster Recovery (DR), backup, failover, and cross-region replication strategies.
• Build comprehensive monitoring, alerting, logging, and observability solutions using Prometheus, Grafana, OpenTelemetry, ELK, and Confluent monitoring tools.
• Optimize cluster performance, troubleshoot production issues, conduct root cause analysis, and implement long-term platform improvements.
• Support Kafka deployments on Kubernetes using Strimzi, Confluent Operator, or managed cloud services including AWS MSK, Azure Event Hubs (Kafka API), and Confluent Cloud.
• Collaborate with application developers, data engineers, DevOps, SRE, and enterprise architects to establish event-driven architecture standards and best practices.
• Conduct architecture reviews, establish platform governance, perform code reviews, and mentor engineers on Kafka development and operations.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related technical discipline.
• 10+ years of professional experience in distributed systems, platform engineering, or infrastructure engineering, including 5+ years of hands-on Apache Kafka or Confluent Platform experience.
• Expert-level knowledge of Kafka internals, including partitions, replication, brokers, ISR, consumer groups, offset management, and performance tuning.
• Extensive experience implementing Kafka security using SASL, SSL/mTLS, ACLs, RBAC, and enterprise authentication mechanisms.
• Strong hands‑on experience with Kafka Connect, Schema Registry, Kafka Streams, ksqlDB, and real-time event streaming architectures.
• Experience implementing High Availability (HA), Disaster Recovery (DR), multi-cluster replication, and cross-region failover strategies.
• Advanced scripting skills using Python, Bash, or Go, along with Infrastructure as Code experience using Terraform, Ansible, or similar tools.
• Strong experience with observability platforms including Prometheus, Grafana, OpenTelemetry, ELK, and distributed tracing solutions.
• Excellent troubleshooting, communication, documentation, stakeholder management, and technical leadership skills.

Preferred Qualifications

• Confluent Certified Administrator or Confluent Certified Developer certification.
• Experience operating Kafka on Kubernetes using Strimzi, Confluent Operator, or similar operators.
• Hands-on experience with managed Kafka services including AWS MSK, Confluent Cloud, Azure Event Hubs (Kafka API), or Google Cloud Managed Kafka.
• Experience with stream processing technologies such as Apache Flink, Apache Spark Structured Streaming, or Apache Beam.
• Knowledge of data governance, schema evolution, metadata management, and data lineage for streaming platforms.

Bright Vision Technologies is an Equal Opportunity Employer.

Equal Employment Opportunity (EEO) Statement

Bright Vision Technologies (BV Teck) is committed to equal employment opportunity (EEO) for all employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, veteran status, or any other protected status as defined by applicable federal, state, or local laws. This commitment extends to all aspects of employment, including recruitment, hiring, training, compensation, promotion, transfer, leaves of absence, termination, layoffs, and recall.

BV Teck expressly prohibits any form of workplace harassment or discrimination. Any improper interference with employees' ability to perform their job duties may result in disciplinary action up to and including termination of employment.