BigData and Hadoop Ecosystems

Tekskills; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Tekskills Bengaluru seeks a Kafka Administrator to manage large-scale Kafka environments. You will install, configure, and maintain Confluent and AWS MSK clusters across on-prem and cloud, and support topics, ACLs, and data flow.

You will partner with development teams, implement security, monitor with Prometheus and Grafana, and drive automation using Ansible, GitOps, and Terraform. Strong troubleshooting and collaboration skills are essential.

Qualifications

• Experience managing Kafka clusters across on-prem and cloud environments.
• Experience administering Confluent Platform components (Brokers, Zookeepers, Schema Registry, RestProxy, KSQL, Connectors, Control Center).
• Proficiency with monitoring/alerting tools and Kubernetes.”],
• job_responsibilities_description and novelty?
• CoT Job Summary?
• Oops

Responsibilities

• Install, configure, and maintain Confluent and AWS MSK Kafka clusters.
• Manage large-scale, multi-node Kafka cluster environments (on-prem and cloud).
• Troubleshoot and resolve issues related to Kafka brokers, topics, and data flow.
• Plan and execute Kafka version upgrades and apply patches.
• Develop and maintain disaster recovery plans.
• Create and maintain documentation.
• Security: Implement and manage security protocols.
• Performance and Monitoring: Monitor clusters; collect metrics; fine-tune configurations.
• Collaboration: Provide guidance to developers; Automation: Automate deployments using Ansible, GitOps, Terraform; Develop automation for Kafka infra.

Skills

Kafka Administration

Troubleshooting

Team collaboration

Tools

Prometheus

Grafana

Dynatrace

Linux

Kubernetes

Ansible

GitOps

Terraform

GitLab

JIRA

Confluent Platform

AWS MSK

Job description

Job Description:

Role descriptions / Expectations from the Role
Experience in other messaging systems such as IBM MQ or Rabbit MQ.
Experience in the Atlassian stack tools Jira, Confluence and Bitbucket (GIT).
Experience in Jenkins automation development.
Experience in the scripting languages shell and python.
Experience in the operating systems Linux (RedHat and/or SuSE) and Windows Server.
Experience in search engines such as Elasticsearch.

Job description: must have
Cluster Management:
Install, configure, and maintain Confluent and AWS MSK Kafka clusters.
Manage large-scale, multi-node Kafka cluster environments (on-prem and cloud).
Troubleshoot and resolve issues related to Kafka brokers, topics, and data flow.
Optimize Kafka configurations for performance, scalability, and fault tolerance.
Plan and execute Kafka version upgrades and apply patches.
Develop and maintain disaster recovery plans.
Create and maintain documentation.

• Security:
  Implement and manage security protocols.
• Performance and Monitoring:
  Monitor Kafka clusters to ensure optimal performance and availability.
  Collect and maintain performance metrics.
  Fine-tune Kafka configurations based on performance data.
• Collaboration:
  Provide guidance and support to developers using Kafka.
  Automation:
  Automate Kafka Cluster installations and topics/acls/connectors deployments using tools like Ansible, terraform, gitops.
  Develop and maintain automation for Kafka infrastructure.

Skills and Experience:

• Kafka Administration
  Strong understanding of Kafka architecture, concepts, and best practices.
  Proven experience in administering Confluent platform Brokers, Zookeepers, Schema registry, Restproxy, Ksql, Connectors and Control Center, Topics, ACLs etc.,
  Experience in administering and supporting AWS MSK Clusters.
  Experience with Kafka AdminClient for managing topics programmatically.
• Technical Skills:
  Experience with monitoring and alerting tools like Prometheus, Grafana, Alertmanager, Dynatrace etc.,
  Experience with Linux/Unix operating systems.
  Knowledge of software containerization and Kubernetes.
  Experience with scripting languages (Python, Shell).
  Automate installations and deployments using tools like Ansible, gitops, terraform etc.,
  Experience with GitLab pipeline.
  Knowledge of Service-Now, JIRA software.
• Good to have:
  Strong troubleshooting skills.
  Excellent communication and collaboration skills.
  Ability to work as part of a team and independently.
  Experience in providing enhanced support to application teams.