Lead Kafka Middleware Engineer

Request Technology; Coppell (TX); On-site; USD 130,000 - 180,000; Full time

Job summary

A prestigious financial institution is looking for a Lead Kafka Middleware Engineer with extensive experience in Kafka administration, infrastructure automation, and CI/CD pipelines. The successful candidate will be responsible for designing and implementing scalable middleware solutions and automating cloud resource provisioning.

Qualifications

• 10+ years of experience in infrastructure middleware administration.
• In-depth expertise in Kafka administration.
• AWS Solutions Architect, CKAD or CKA certifications preferred.

Responsibilities

• Design and manage scalable Kafka clusters.
• Develop and maintain IaC using Ansible and Terraform.
• Implement CI/CD pipelines using GitHub, Jenkins, and more.

Skills

IaC tools

Cloud automation

CI/CD tools

Scripting skills

Troubleshooting

Networking principles

Monitoring tools

Kafka administration

AWS Solutions Architect

Container orchestration

Education

Bachelor's degree in Computer Science, Engineering or related field

Tools

Ansible

Terraform

Kubernetes

GitHub

Jenkins

AWS

Azure

Rancher

Job description

Position is bonus eligible.

Prestigious Financial Institution is currently seeking a Lead Kafka Middleware Engineer with deep expertise in Kafka administration, infrastructure as code (IaC), cloud automation, container orchestration and CI/CD pipelines. The ideal candidate will be responsible for designing, implementing, and maintaining robust and scalable middleware solutions, ensuring high availability, performance, and security. Candidate will play a crucial role in automating infrastructure provisioning, deployments, and operations, enabling our organization to rapidly deliver and scale applications.

Responsibilities:

• Design, implement and manage highly available and scalable Kafka clusters.
• Monitor Kafka performance, troubleshoot issues and optimize configurations.
• Develop and maintain IaC using Ansible and Terraform for infrastructure provisioning and configuration Management.
• Create and maintain reusable IaC modules.
• Design and implement cloud-based infrastructure solutions on AWS and Azure.
• Automate cloud resource provisioning, scaling and management using cloud-native tools and services.
• Deploy and Manage containerized applications using Kubernetes and Rancher
• Troubleshoot container-related issues and optimize container performance.
• Design, implement and maintain CI/CD pipelines using tools like GitHub, Artifactory, Harness and Jenkins.
• Automated the build, test and deployment of middleware components.
• Integrate IaC and container technologies into CI/CD pipelines.
• Document all processes and procedures.
• Work with development teams to ensure smooth deployments.

Qualifications:

• Strong proficiency in IaC tools, specifically Ansible, Terraform and bash scripting.
• Extensive experience with cloud automation and provisioning on AWS and Azure.
• Proficiency in CI/CD tools, including GitHub, Artifactory, Harness and Jenkins.
• Strong scripting skills in languages like Python and Bash.
• Excellent troubleshooting and problem-solving skills
• Understanding of networking principles.
• Experience with monitoring tools like Splunk, Splunk OTEL, Prometheus and Grafana.
• Kafka, Ansible, Terraform, Bash, Kubernetes, Rancher, GitHub, Artifactory, Harness, Jenkins, AWS, Azure, CI/CD, IaC, Automated Cloud Provisioning
• Bachelors degree in Computer Science, Engineering or a related field (or equivalent experience)
• 10+ years of experience in infrastructure middleware administration.
• In-depth expertise in Kafka administration, including cluster management, performance tuning, and security.
• AWS Solutions Architect, CKAD or CKA certifications preferred.