Principal Kafka Engineer

Golden Technology; Cincinnati (OH); On-site; USD 140,000 - 190,000; Full time

Job summary

Golden Technology in the United States is seeking a Principal Kafka Engineer to lead the management, scaling, and optimization of our event-streaming ecosystem. You will apply OO principles, automate cloud/on-prem clusters, and enforce ITSM processes to ensure reliable service delivery.

The role requires deep Kafka expertise, strong Java/Go/Python development skills, and experience with Docker/Kubernetes in a production, global setting. Collaboration with distributed teams is essential.

Qualifications

• 3–5+ years Kafka administration experience and 10+ years in software development or systems engineering.
• Experience with Apache Kafka/Confluent Platform incl. Schema Registry, Kafka Connect, Avro, and KSQL.
• Proficient in Cluster Management — Zookeeper, KRaft.
• Strong OO programming skills: Java, Go, Python.
• Hands-on with IaC tools (Terraform, Ansible, Salt).
• Java backend development with Spring Boot.
• Solid software design patterns knowledge.
• Experience with Docker, Kubernetes; DevOps CI/CD and GitHub Actions.
• Knowledge of Domain Driven Design and Enterprise Application Integration.
• Observability/telemetry/APM tools (e.g., dynaTrace) and cloud/networking basics.
• Ability to lead in a globally distributed team and communicate well.
• Familiarity with ITSM/ITIL and Agile methodologies.

Responsibilities

• Kafka Administration: Deploy, configure, and maintain highly available Kafka clusters across on-premises and cloud environments.
• OOP-Based Development and Automation: Build reusable frameworks, automation tools, and Java APIs for provisioning and onboarding.
• ITSM Integration: Manage lifecycle via ITSM processes to reduce downtime and maintain SLA.
• Security and Compliance: Implement and maintain security controls and authentication mechanisms.
• Performance Optimization: Monitor health with Prometheus, Grafana, and dynaTrace; perform capacity planning.
• Collaboration: Guide DevOps and app teams on event-driven architecture and performance best practices.

Skills

Kafka

Java

Go

Python

Spring Boot

OO Design

Architecture

DevOps

CI/CD

GitHub Actions

ITSM/ITIL

Docker

Kubernetes

Terraform

Ansible

Zookeeper

KRaft

dynaTrace

Prometheus

Grafana

Tools

Terraform

Ansible

Salt

Docker

Kubernetes

GitHub Actions

Prometheus

Grafana

dynaTrace

Job description

We are seeking a Principal Kafka Engineer to lead the management, scaling, and optimization of our enterprise event-streaming ecosystem. The ideal candidate brings deep expertise in Apache/Confluent Kafka, applies object-oriented programming principles to advanced automation, and uses ITSM practices to support reliable, high-quality service delivery.

Required Qualifications:

• Experience: 3-5+ years of Kafka administration experience and 10+ years in software development or systems engineering.
• Technical Skills:
• Expertise in Apache Kafka or Confluent Platform, including Schema Registry, Kafka Connect, Avro, and KSQL.
• Proficient in Cluster Management - Zookeeper, KRaft
• Strong object-oriented programming skills; Java, Go, Python
• Hands-on experience with Infrastructure as Code tools such as Terraform, Ansible, or Salt.
• Java backend development experience with Spring Boot.
• Solid understanding of software design patterns.
• Strong knowledge of architecture and design concepts, with the ability to create and maintain architecture artifacts.
• Proficient in containerization technology such as Docker, Kubernetes etc.
• Expertise in DevOps, CI/CD, and GitHub Actions.
• Expertise in Domain Driven Design and Enterprise Application Integration.
• Experience with observability, telemetry, and APM tools such as dynaTrace.
• Good understanding of networking, storage, virtualization, and other infrastructure architecture.
• Self-starter with a goal-oriented mindset.
• Excellent communication, negotiation, and problem-solving skills.
• Expertise in supporting a Highly Available Low Latent Enterprise Application in Production environment(s).
• Experience working with globally distributed team(s).
• Frameworks: Solid understanding of ITSM/ITIL frameworks and Agile methodologies.

Desired Qualifications:

• Front-end development experience with Angular, React (preferred), or other TypeScript-based frameworks.
• Expertise in FinOps.
• Experience working in platform teams.

Preferred Certifications:

• Cloud provider certifications, such as Azure Certified Solutions Architect.
• Confluent Certified Administrator for Apache Kafka (CCAAK).
• TOGAF

Key Responsibilities:

• Kafka Administration: Deploy, configure, and maintain highly available Kafka clusters across on-premises and cloud environments, including Azure and GCP. Manage topics, partitions, and replication to ensure platform reliability.
• OOP-Based Development and Automation: Design and build reusable frameworks, automation tools, and APIs using Java to simplify cluster provisioning, monitoring, and self-service onboarding.
• ITSM Integration: Manage the platform lifecycle through ITSM processes, including incident, problem, and change management, to reduce downtime and maintain SLA compliance.
• Security and Compliance: Implement and maintain security controls and authentication mechanisms.
• Performance Optimization: Monitor cluster health with tools such as dynaTrace, Prometheus, and Grafana. Perform capacity planning and tuning to support high-throughput data pipelines.
• Collaboration: Partner with DevOps and application teams to provide guidance on event-driven architecture and producer/consumer performance best practices.

NOTES:

• Top 3 skills for each req : Java (Object Oriented Programming), Kafka, Software Architecture, Vulnerability Management
• Soft Skills needed : Self Starter, Driven, Negotiation, Effective Communication
• Are there any automatic disqualifiers? - Someone who can't drive things, understand what is being asked, unable to prioritize, support production issues, mitigate security risks
• Team dynamics & make-up (size, locations) - right now it is a 12-member team, with a mix of onsite, offshore team