Senior Kafka Engineer - Cloud & On-Prem Ops

Artifintech; Atlanta (GA); On-site; USD 140,000 - 190,000; Full time

Job summary

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking an experienced Kafka administrator to install, configure, and manage distributed Kafka clusters across on-prem and cloud environments. You will monitor health, performance, and capacity, manage topics and retention, and implement robust security controls while collaborating with platform teams on real-time data use cases.

The role requires deep knowledge of Kafka internals, Zookeeper, and Kafka Connect, plus strong Linux, scripting, and DevOps

Qualifications

• Requires a Bachelor's or Master's degree in computer science, engineering, or a related field.
• Minimum 10+ years of software engineering experience.
• 3–5 years of Kafka administration in enterprise environments.
• Strong understanding of Kafka internals, Zookeeper, and Kafka Connect.
• Experience with Kafka in cloud-native environments (AWS MSK, Confluent Cloud, or Kubernetes).
• Proficiency in Linux, shell scripting, and monitoring/logging tools.
• Familiarity with CI/CD pipelines and DevOps practices.

Responsibilities

• Install, configure, and manage Kafka clusters (on-prem/cloud).
• Monitor cluster health, performance, and resource utilization.
• Manage Kafka topics, partitions, replication, and retention policies.
• Monitor Kafka performance, availability, and capacity using Prometheus, Grafana, and Confluent Control Center.
• Implement security controls (authentication, authorization, encryption).
• Implement security best practices including TLS, Kerberos, and RBAC.
• Troubleshoot and resolve issues related to Kafka brokers, producers, and consumers.
• Collaborate with platform architects and developers to support real-time data use cases.
• Automate operational tasks using scripting and infrastructure-as-code tools
• Support upgrades, patching, and disaster recovery planning

Skills

Linux

Shell scripting

Monitoring

CI/CD

DevOps

Education

Bachelor's or Master's in CS/Engineering

Tools

Prometheus

Grafana

Confluent Control Center

Zookeeper

Kubernetes

AWS MSK

Job description

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking an experienced Kafka administrator to install, configure, and manage distributed Kafka clusters across on-prem and cloud environments. You will monitor health, performance, and capacity, manage topics and retention, and implement robust security controls while collaborating with platform teams on real-time data use cases.

The role requires deep knowledge of Kafka internals, Zookeeper, and Kafka Connect, plus strong Linux, scripting, and DevOps