Kafka Admin/ Engineer

TekWissen LLC; New York (NY); Hybrid; USD 96,432,000 - 151,536,000; Full time

Job summary

TekWissen Group is seeking a Kafka Admin/Engineer for a hybrid, six-month assignment in New York, NY. You will design, deploy, and maintain Kafka clusters in production and support the Linux-based infrastructure underpinning our real-time data platform.

Responsibilities include upgrades, security controls, monitoring with Prometheus/Grafana, automation with IaC tools (Ansible, Terraform), and collaboration with engineering to onboard new use cases while ensuring reliability and compliance.

Qualifications

• 5+ years of Kafka administration experience, ideally with Confluent Kafka.
• Experience with Linux-based infrastructures and scripting.

Responsibilities

• Design, implement, and maintain highly available Kafka clusters in production.
• Perform Kafka upgrades and patching with minimal downtime.
• Administer security protocols and access controls for the platform.
• Monitor cluster health using Prometheus and Grafana.
• Automate provisioning and configuration using IaC/Ansible/Terraform.
• Collaborate with engineering to onboard new Kafka use cases.

Skills

Kafka administration

Linux

Python

Bash scripting

OpenShift

Kubernetes

Terraform

Ansible

Kafka Connect

Education

BE/B Tech/MCA

Tools

Docker

Prometheus

Grafana

Zookeeper

Schema Registry

Job description

Overview

TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client is a global professional services firm specializing in digital transformation, business process management (BPM), analytics, and AI-driven solutions. The company partners with organizations to improve efficiency, transform operations, and drive business outcomes through a combination of process expertise, technology, and data insight.

Job Title

Kafka Admin/ Engineer

Location

New York, NY

Duration

6 Months

Job Type

Temporary Assignment

Work Type

Hybrid

Job Description

Job Summary

The ideal candidate will be responsible for designing, deploying, maintaining, and scaling Kafka clusters in mission-critical environments, while also supporting the Linux-based infrastructure that forms the foundation of our real-time data platform.

Responsibilities

• Design, implement, and maintain highly available Kafka clusters in production environments.
• Perform Kafka upgrades and patching ensuring minimal downtime and seamless transitions.
• Administer security protocols and access controls to protect the platform.
• Monitor cluster health and performance using our established monitoring stack (Prometheus, Grafana)
• Troubleshoot and resolve cluster-related incidents such as broker outages, replication lag.
• Develop and enhance automation scripts to streamline operational tasks and reduce manual intervention.
• Assist with planning and managing cluster resources to support future growth.
• Develop Kafka topics, partitions, ACLs, schema registry, and optimize throughput and reliability.
• Manage a tune of Kafka Connect, Kafka Streams, and Mirror Maker as needed for data pipelines.
• Ensure platform reliability and performance on enterprise-grade Linux systems (RHEL, Ubuntu).
• Automate Kafka provisioning, configuration, and monitoring using Infrastructure as Code (IaC) and configuration management tools (e.g., Ansible, Terraform).
• Deep knowledge of Kafka ecosystem components like ZooKeeper, Kafka Connect, Schema Registry, MirrorMaker, KRaft (Kafka without ZooKeeper).
• Build custom Kafka tooling and dashboards for producers and consumers, monitor lag and health metrics.
• Work with engineering teams to onboard new Kafka use cases and ensure security and compliance standards.
• Perform root cause analysis and resolution of platform-level incidents and outages.Develop and maintain Linux shell scripts for automation of deployments, monitoring, and maintenance tasks.

Qualifications

• BE/B Tech/MCA
• Excellent written and verbal communication skills

Qualifications/ Skills

• Working experience with OpenShift Kubernetes Cloud services to deploy Kafka clusters using vendor provided helm charts & docker images
• Kafka Resiliency dc/dr setup & observability for Kafka nodes
• Data ingestion & streaming pipeline setup for other data platforms to Kafka & Kafka to data sources like Solr
• Five or more years of Kafka administration experience, ideally with Confluent Kafka
• Proficiency in Linux, Python, or Bash scripting
• Hands-on knowledge of Kafka monitoring tools and experience with security protocols.

TekWissen Group is an equal opportunity employer supporting workforce diversity.