Senior Software Engineer-Senior Cloud Data Engineer

Nestdigital; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

You will monitor health metrics, optimize performance, implement security controls, and participate in DR tests and platform upgrades. Strong scripting and problem-solving skills are essential for production reliability.

Qualifications

• Hands-on admin experience with Confluent Platform components (Kafka Brokers, Schema Registry, Kafka Connect, Control Center, ksqlDB/Flink).
• Experience in Linux server management on cloud/on-prem environments (Azure/AWS/IaaS).
• Proficient in automation using Ansible, Shell scripting, or Python.

Responsibilities

• Administer, monitor, and support Confluent Kafka Platform environments for high availability, performance, and security.
• Install, configure, upgrade, patch, and maintain Kafka Brokers, ZooKeeper/KRaft, Schema Registry, REST Proxy, and related services.
• Deploy, manage, and troubleshoot Kafka Connect connectors including CDC, JDBC, S3/ADLS, IBM MQ, and REST connectors.

Skills

Kafka Connect

Confluent Kafka

Ansible

Infrastructure As Code

Job description

General

Posted on Sep 11, 2026

Share on

Minimum Required Experience : 4 years

Contract

Skills

Kafka Connect

Confluent Kafka

Ansible

Infrastructure As Code

Description

KEY ACCOUNTABILITIES:

• Administer, monitor, and support Confluent Kafka Platform environments, ensuring high availability, performance, scalability, and security.
• Install, configure, upgrade, patch, and maintain Kafka brokers, ZooKeeper/KRaft, Schema Registry, Kafka Connect, Control Center, Flink and REST Proxy.
• Deploy, manage, and troubleshoot Kafka Connect source and sink connectors including CDC, JDBC, S3/ADLS, IBM MQ, and REST connectors.
• Manage platform infrastructure on PAAS/IaaS environments (Azure, AWS, or On-Premises Linux servers), including capacity planning and performance tuning.
• Automate Kafka platform provisioning, deployment, and operational activities using Ansible and Infrastructure as Code practices.
• Monitor platform health, consumer lag, broker performance, connector status, and proactively resolve incidents and production issues.
• Implement security controls including SSL/TLS, RBAC, LDAP/AD integration, Secrets Management, and encryption standards.
• Perform disaster recovery testing, backup validation, platform upgrades, and infrastructure maintenance activities.
• Raise and manage ServiceNow incidents, service requests, change requests, and support audit/compliance requirements.
• Collaborate with application, infrastructure, cloud, and data engineering teams to onboard and support enterprise streaming use cases.

. FRAMEWORKS, BOUNDARIES, & DECISION MAKING AUTHORITY:

• Functions within the framework and boundaries of Group policies as well as overall organisational and governance frameworks.
• Authorised to take decisions as per the approved authorisation matrix.
• 4-6 years of experience in Confluent Platform administration and support.
• Strong experience with Confluent Platform components including Kafka Brokers, Schema Registry, Kafka Connect, Control Center, and ksqlDB/Flink.
• Hands-on experience with Flink for stream processing and real-time data pipelines.
• Experience managing Linux-based infrastructure in Azure, AWS, or other IaaS platforms.
• Strong automation and scripting experience using Ansible, Shell Scripting, or Python.
• Experience with SSL, RBAC, LDAP, authentication, authorization, and Kafka security best practices.
• Knowledge of monitoring tools such as Prometheus, Grafana, Splunk, or ELK.
• Experience with CI/CD pipelines and infrastructure automation.
• Strong troubleshooting, problem-solving, and stakeholder communication skills.

Experience in implement application self-healing solutions through proactive and reactive automated measures

Other jobs that might interest you

Senior Software Engineer-Kubernetes Platform Engineer