Kafka Architect

TechDigital Group; Minot (ND); On-site; USD 100,000 - 130,000; Full time

Job summary

A technology solutions provider is seeking a Kafka Architect in Minot, North Dakota. The role involves consulting teams on Kafka usage, building and supporting Kafka clusters, and automating operational processes. Candidates should have a strong foundation in DevOps tools and at least 2 years of experience in Kafka clustering and administration. The ideal candidate is also familiar with cloud environments and agile methodologies. Competitive compensation is offered.

Qualifications

• 2+ years of experience with Kafka clustering & administration.
• 2+ years of experience with Infrastructure-as-Code best practices.
• Experience in developing automated processes around Kafka.

Responsibilities

• Consult with teams on leveraging Kafka in their pipelines.
• Architect, build, and support Kafka clusters via IaC.
• Automate operational processes where possible.

Skills

DevOps

Kafka

Splunk

Ansible

Grafana

Tools

Chef

Jenkins

Artifactory

Kubernetes

Docker

Job description

Kafka Architect

Mandatory Skills - DevOps, Kafka, Splunk, Ansible, Grafanna

Responsibilities

• Consult with inquiring teams on how to leverage Kafka within their pipelines
• Architect, Build and Support existing and new Kafka clusters via IaC
• Partner with Splunk teams to route traffic through Kafka by utilizing open-source agents and collectors
• deployed via Chef
• Remediate any health issues within Kafka
• Automate (where possible) any operational processes on the team
• Create new and/or update monitoring dashboards and alerts as needed
• Manage a continuous improvement / continuous development (CI/CD) pipeline
• Perform PoC's on new components to expand/enhance team's Kafka offerings

Skills & Qualifications

• 2+ years of experience with Kafka clustering & administration
• 2+ years of experience building, deploying, and supporting multiple Kafka clusters using IaC (Infrastructure-as-Code) best practices
• Experience developing automated processes within and around Kafka to help supplement the service
• Experience working with multiple teams to build and architect data pipeline solutions where Kafka will be involved.
• Experience with Linux/Unix and system management
• IaC (Infrastructure-as-Code) experience with Virtual/Physical Servers using Chef, Ansible, Jenkins, Artifactory, etc.
• Understanding of Git workflows, continuous improvement / continuous development (CI/CD) conceptsStrong verbal and written communication skills
• Strong technical acumen
• Strong analytical skills
• Experience working in Agile/Lean methodologies

Preferred Qualifications

• Knowledge and experience with Splunk, Elastic, Kibana and Grafana
• Knowledge and experience with log collection agents such as Open-Telemetry, Fluent Bit, FluentD, Beats and LogStash.
• Knowledge and experience with Kubernetes / Docker
• Knowledge and experience with Kafka-Connect
• Knowledge and experience with AWS or Azure
• Knowledge and experience with Streaming Analytics