Consultant - Database Administrator

Darwinbox Digital Solutions Pvt. Ltd.; India; On-site; INR 3,000,000 - 5,000,000; Full time

Job summary

Darwinbox Digital Solutions Pvt. Ltd. is seeking a Data Platform Engineer - Streaming Platforms to administer and support enterprise Apache Kafka environments. You will install, configure, monitor, tune, and upgrade Kafka clusters handling real-time streaming workloads.

The role collaborates with application, data engineering, infrastructure, security, and production teams to ensure stable, secure, and highly available streaming services. 8–13 years IT experience required.

Qualifications

• Hands-on experience with Apache Kafka in production or enterprise environments.
• Experience with Kafka cluster installation, configuration, administration, maintenance, and upgrades.
• Strong knowledge of topics, partitions, replication, brokers, producers, consumers, offsets, and consumer groups.
• Experience monitoring, troubleshooting, and performance tuning Kafka clusters and streaming workloads.
• Knowledge of Kafka security, including SSL/TLS, SASL authentication, and ACL based authorization.
• Working knowledge of Linux/Unix environments and Shell scripting.

Responsibilities

• Install, configure, administer, maintain, and upgrade Kafka clusters across enterprise environments.
• Monitor Kafka platform health, availability, capacity, and performance; troubleshoot issues and implement tuning recommendations.
• Create, configure, and manage Kafka topics, partitions, replication factors, retention settings, and other resources.
• Manage and troubleshoot producers, consumers, offsets, and consumer groups for reliable message processing.
• Implement and maintain Kafka security using SSL/TLS, SASL, and ACLs.
• Support production Kafka environments, participate in incident troubleshooting, perform root-cause analysis, and assist with service restoration.
• Analyze logs (Kafka, broker, OS, apps) to identify and resolve production issues.
• Perform routine maintenance, patching, upgrades, capacity planning, and health checks.
• Support high availability, replication, resiliency, backup/recovery, and disaster-recovery requirements for streaming platforms.
• Use Linux/Unix administration and Shell scripting to support platform operations and automate tasks.
• Maintain operational docs, runbooks, configuration standards, troubleshooting procedures, and inventories.
• Collaborate with application and engineering teams to onboard new Kafka use cases and resolve integration issues.
• Support Kafka environments using ZooKeeper and/or KRaft architecture and participate in modernization activities.

Skills

Kafka install

Monitoring & tuning

Topics & replication

Kafka security

Linux/Unix

Production support

Tools

Confluent Kafka

Kafka Connect

Prometheus & Grafana

CI/CD / IaC scripting

Job description

As applicable based on functional alignment

Direct Reports:

Position Summary:

The Data Platform Engineer - Streaming Platforms is responsible for the administration, reliability, security, and operational support of enterprise Apache Kafka environments. This role is intended for an experienced Kafka Administrator with 4-8 years of overall IT experience and hands-on experience administering Apache Kafka platforms. The engineer will install, configure, administer, monitor, troubleshoot, tune, and upgrade Kafka clusters that support business-critical real-time streaming workloads. The role works closely with application, data engineering, infrastructure, security, and production support teams to maintain stable, secure, and highly available streaming services.

Responsibilities:

• Install, configure, administer, maintain, and upgrade Apache Kafka clusters across enterprise environments.
• Monitor Kafka platform health, availability, capacity, and performance; troubleshoot platform issues and implement performance-tuning recommendations.
• Create, configure, and manage Kafka topics, partitions, replication factors, retention settings, and other cluster resources.
• Manage and troubleshoot producers, consumers, offsets, and consumer groups to support reliable message processing.
• Implement and maintain Kafka security using SSL/TLS, SASL, and Access Control Lists (ACLs).
• Support production Kafka environments, participate in incident troubleshooting, perform root-cause analysis, and assist with service restoration.
• Analyze Kafka, broker, operating-system, and application logs to identify and resolve production issues.
• Perform routine platform maintenance, patching, upgrades, capacity planning, and operational health checks.
• Support high availability, replication, resiliency, backup/recovery, and disaster-recovery requirements for streaming platforms.
• Use Linux/Unix administration and Shell scripting to support platform operations and automate repetitive administrative tasks.
• Maintain operational documentation, runbooks, configuration standards, troubleshooting procedures, and platform inventories.
• Collaborate with application and engineering teams to onboard new Kafka use cases and resolve platform integration issues.
• Support Kafka environments using ZooKeeper and/or KRaft architecture and participate in platform modernization activities.

Education & Experience:

8-13 years of overall Information Technology experience.

• Hands-on experience with Apache Kafka administration in production or enterprise environments.
• Experience with Kafka cluster installation, configuration, administration, maintenance, and upgrades.
• Strong knowledge of Kafka topics, partitions, replication, brokers, producers, consumers, offsets, and consumer groups.
• Experience monitoring, troubleshooting, and performance tuning Kafka clusters and streaming workloads.
• Hands-on knowledge of Kafka security, including SSL/TLS, SASL authentication, and ACL based authorization.
• Working knowledge of Linux/Unix environments and Shell scripting.
• Experience supporting production systems and troubleshooting incidents in business critical environments.
• Knowledge of Apache ZooKeeper and Kafka KRaft architecture.
• Effective written and verbal communication skills and the ability to collaborate across technical teams.

Must Have Skills

• Kafka cluster installation, configuration, administration, and upgrades
• Monitoring, troubleshooting, and performance tuning
• Topics, partitions, replication, and consumer group management
• Kafka security - SSL/TLS, SASL, and ACLs
• Linux/Unix and Shell scripting
• Production support and incident troubleshooting

Nice to Have Skills

• Hands-on experience with Confluent Kafka or Confluent Platform.
• Experience with Kafka Connect and connector administration.
• Experience with Prometheus and Grafana for Kafka monitoring, metrics, dashboards, and alerting.
• Experience with automation, CI/CD, Infrastructure-as-Code, or scripting used to operate streaming platforms.

Licenses, Certifications & Training :

Knowledge, Skills, Abilities, Behaviors: