Kafka Admin

Artifintech; Atlanta (GA); On-site; USD 140,000 - 190,000; Full time

Job summary

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is seeking an experienced Kafka administrator to install, configure, and manage distributed Kafka clusters across on-prem and cloud environments. You will monitor health, performance, and capacity, manage topics and retention, and implement robust security controls while collaborating with platform teams on real-time data use cases.

The role requires deep knowledge of Kafka internals, Zookeeper, and Kafka Connect, plus strong Linux, scripting, and DevOps

Qualifications

• Requires a Bachelor's or Master's degree in computer science, engineering, or a related field.
• Minimum 10+ years of software engineering experience.
• 3–5 years of Kafka administration in enterprise environments.
• Strong understanding of Kafka internals, Zookeeper, and Kafka Connect.
• Experience with Kafka in cloud-native environments (AWS MSK, Confluent Cloud, or Kubernetes).
• Proficiency in Linux, shell scripting, and monitoring/logging tools.
• Familiarity with CI/CD pipelines and DevOps practices.

Responsibilities

• Install, configure, and manage Kafka clusters (on-prem/cloud).
• Monitor cluster health, performance, and resource utilization.
• Manage Kafka topics, partitions, replication, and retention policies.
• Monitor Kafka performance, availability, and capacity using Prometheus, Grafana, and Confluent Control Center.
• Implement security controls (authentication, authorization, encryption).
• Implement security best practices including TLS, Kerberos, and RBAC.
• Troubleshoot and resolve issues related to Kafka brokers, producers, and consumers.
• Collaborate with platform architects and developers to support real-time data use cases.
• Automate operational tasks using scripting and infrastructure-as-code tools
• Support upgrades, patching, and disaster recovery planning

Skills

Linux

Shell scripting

Monitoring

CI/CD

DevOps

Education

Bachelor's or Master's in CS/Engineering

Tools

Prometheus

Grafana

Confluent Control Center

Zookeeper

Kubernetes

AWS MSK

Job description

• Install, configure, and manage Kafka clusters (on-prem/cloud).
• Monitor cluster health, performance, and resource utilization.
• Manage Kafka topics, partitions, replication, and retention policies.
• Monitor Kafka performance, availability, and capacity using tools like Prometheus, Grafana, and Confluent Control Center.
• Implement security controls (authentication, authorization, encryption).
• Implement security best practices including TLS, Kerberos, and RBAC.
• Troubleshoot and resolve issues related to Kafka brokers, producers, and consumers.
• Collaborate with platform architects and developers to support real-time data use cases.
• Automate operational tasks using scripting and infrastructure-as-code tools
• Support upgrades, patching, and disaster recovery planning

Required Qualifications:

• Bachelor's or master's degree in computer science, Engineering, or related field.
• Minimum 10+ years of experience in software engineering,
• 3-5 years of experience in Kafka administration in enterprise environments.
• Strong understanding of Kafka internals, Zookeeper, and Kafka Connect.
• Experience with Kafka in cloud-native environments (AWS MSK, Confluent Cloud, or self-managed on Kubernetes).
• Proficiency in Linux, shell scripting, and monitoring/logging tools.
• Familiarity withCI/CD pipelines and DevOps practices.

ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will ARTIFICIAL INTELLIGENCE TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract