Kafka Administrator

Veriipro; Englewood (CO); On-site; USD 130,000 - 170,000; Full time

Job summary

Veriipro in Colorado is seeking an experienced Kafka Administrator to manage large-scale production Kafka environments. You will design, deploy, and operate highly available Kafka clusters across on-prem, AWS MSK, and Confluent Cloud, with emphasis on reliability and minimal downtime.

The ideal candidate has 8–10 years of IT experience, strong SRE/DevOps background, and hands-on expertise with Kafka ecosystem tools such as Kafka Connect, Schema Registry, MirrorMaker, and Confluent Replicator.

Qualifications

• 8–10 years of IT experience in enterprise environments.
• 4+ years of production Kafka operations at scale in an SRE/DevOps context.
• Strong experience with Apache Kafka administration and production operations.
• Hands-on AWS MSK operations, configuration, and integration.
• Experience with Kafka Connect, Schema Registry, MirrorMaker, and Confluent Replicator.
• Solid Linux admin and networking fundamentals (TCP/IP, DNS, load balancing).
• Proven ability to perform hardware refreshes, major cluster rebuilds, and migrations with minimal downtime.
• Experience with Kafka performance tuning, capacity planning, and troubleshooting.
• Knowledge of HA, replication, disaster recovery, monitoring, and best practices.
• Ability to create technical documentation, migration plans, and runbooks.
• Strong communication and collaboration skills.

Responsibilities

• Design, deploy, configure, and maintain highly available Kafka clusters across on-prem, cloud, and managed environments.
• Administer Kafka topics, partitions, quotas, retention policies, replication settings, and consumer groups.
• Manage Kafka ecosystem components: Kafka Connect, Schema Registry, MirrorMaker, Confluent Replicator, REST Proxy.
• Execute upgrades, patching, hardware refreshes, cluster rebuilds, and migrations with minimal downtime.
• Perform Kafka performance tuning, throughput testing, and benchmarking across producers, consumers, brokers, and controllers.
• Optimize batching, acks, ISR, and cluster health.
• Develop capacity plans, partitioning strategies, and sizing models.
• Integrate Kafka with AWS MSK and other enterprise systems.
• Implement monitoring, alerting, backup, recovery, and DR procedures.
• Apply Linux and networking fundamentals to troubleshoot infrastructure issues.
• Develop architecture diagrams, runbooks, and migration documentation.
• Conduct knowledge transfer sessions and ensure smooth handoff at milestones.

Skills

Kafka administration

SRE/DevOps

AWS MSK

Linux administration

Networking fundamentals

Cluster upgrades

Monitoring & DR

Documentation skills

Collaboration skills

Tools

Kafka Connect

Schema Registry

MirrorMaker

Confluent Replicator

REST Proxy

Job description

We are seeking an experienced Kafka Administrator with strong expertise in Apache Kafka, Confluent technologies, AWS MSK, and large-scale production environments. The ideal candidate will have a background in Site Reliability Engineering (SRE) or DevOps, with hands-on experience managing highly available Kafka clusters, performing major cluster rebuilds and hardware refreshes, and executing migrations with minimal downtime.

Key Responsibilities

• Design, deploy, configure, and maintain highly available Apache Kafka clusters across on-premises, cloud, and managed environments, including AWS MSK and Confluent Cloud.
• Administer Kafka topics, partitions, quotas, retention policies, replication settings, and consumer groups to ensure optimal performance and cost efficiency.
• Manage Kafka ecosystem components, including Kafka Connect, Schema Registry, MirrorMaker, Confluent Replicator, and REST Proxy.
• Execute Kafka upgrades, patching, hardware refreshes, cluster rebuilds, and migrations while minimizing downtime and operational risks.
• Perform Kafka performance tuning, throughput testing, and benchmarking across producers, consumers, brokers, and controllers.
• Optimize batching, compression, acknowledgments (acks), in-sync replicas (ISR), and cluster health.
• Develop capacity plans, partitioning strategies, infrastructure sizing models, and cluster right-sizing recommendations.
• Integrate Apache Kafka with AWS MSK and other enterprise systems.
• Implement monitoring, alerting, troubleshooting, backup and recovery, and disaster recovery procedures.
• Apply strong Linux and networking fundamentals, including TCP/IP, DNS, and load balancing, to troubleshoot infrastructure and connectivity issues.
• Develop architecture diagrams, operational standards, runbooks, and migration documentation.
• Conduct knowledge transfer sessions and ensure smooth documentation handoff at key milestones and project completion.

Required Skills and Qualifications

• 8–10 years of overall relevant IT experience.
• 4+ years of hands-on experience operating Kafka in production at scale within an SRE or DevOps environment.
• Strong experience with Apache Kafka administration and production operations.
• Hands-on experience with AWS MSK operations, configuration, and integration.
• Practical experience with Kafka Connect, Schema Registry, MirrorMaker, and Confluent Replicator.
• Strong Linux administration and networking fundamentals, including TCP/IP, DNS, and load balancing.
• Proven experience executing hardware refreshes, major cluster rebuilds, and migrations with minimal downtime.
• Experience with Kafka performance tuning, capacity planning, cluster sizing, and troubleshooting.
• Knowledge of high availability, replication, disaster recovery, monitoring, and operational best practices.
• Ability to create technical documentation, migration plans, architecture diagrams, and operational runbooks.
• Strong communication and collaboration skills.