Lead I - Kafka Support Engineer

UST; Bengaluru; On-site; INR 1,500,000 - 2,400,000; Full time

Job summary

UST is seeking an experienced L3 Support Data Engineer to own real-time data pipelines and optimize the AWS-Kafka data stack in Bengaluru. The role emphasizes deep-dive troubleshooting, performance tuning, and reliable data delivery.

The candidate should have 5+ years in data or SRE roles, strong Python/Java/SQL skills, and hands-on experience with MSK or Confluent, Glue, Lambda, and S3. Excellent communication and collaboration are essential.

Qualifications

• 5+ years in data engineering or related roles.
• Strong Kafka and AWS data stack experience.
• Proficiency in Python, SQL, and Java.
• Experience with IaC and CI/CD.
• Excellent problem-solving and communication skills.

Responsibilities

• Provide L3 incident resolution for real-time data pipelines.
• Develop, maintain, and debug real-time data pipelines.
• Configure and troubleshoot Kafka connectors, brokers, topics, and schema registries.
• Design and optimize ETL/ELT workflows.
• Monitor and optimize AWS data lake/warehouse solutions.
• Implement observability and monitoring practices.
• Uphold data security and governance standards.
• Communicate findings to engineering and leadership during incidents.

Skills

Kafka

Python

Java

SQL

AWS

Incident Mgmt

Education

Bachelor's degree in CS/EE

Tools

MSK

Confluent

Glue

S3

CloudWatch

Job description

Job Summary

Role: L3 Support Data Engineer (AWS, Kafka)

Responsibilities

• Advanced Troubleshooting Incident Management: Provide Tier-3 (L3) operational support and deep-dive troubleshooting for real-time data pipelines, distributed systems, and production bottlenecks using Apache Kafka (MSK or Confluent) and AWS services.
• Pipeline Development Maintenance: Develop, maintain, and debug real-time data pipelines to ensure continuous, reliable data delivery.
• Kafka Ecosystem Management: Configure, manage, and troubleshoot Kafka connectors, producers, consumers, brokers, topics, and schema registries to guarantee seamless data flow and integration across enterprise systems.
• ETL/ELT Workflow Optimization: Support, design, and implement scalable ETL/ELT workflows capable of processing large volumes of data efficiently, resolving performance degradation issues.
• AWS Data Stack Optimization: Monitor, troubleshoot, and optimize data lake and data warehouse solutions leveraging AWS services including Lambda, S3, and Glue.
• Observability Reliability: Implement and enhance robust monitoring, automated testing, and observability practices (metrics, logs, traces) to proactively identify platform vulnerabilities and ensure reliability.
• Security Governance: Uphold stringent data security, governance, and compliance standards across all data operations and support workflows.
• Cross-Functional Collaboration: Communicate technical findings clearly to both engineering peers and leadership during critical incident resolutions and post-mortems.

Requirements

• Experience: Minimum of 5 years of experience in data engineering, software engineering, or high-tier technical support/SRE roles within distributed environments.
• Core Technical Expertise: Proven expertise with Apache Kafka and the modern AWS data stack (MSK, Glue, Lambda, S3, CloudWatch, etc.).
• Programming Proficiency: Proficient in coding, debugging, and code review with Python, SQL, and Java (Java strongly preferred). Must demonstrate the flexibility to write and troubleshoot code in both Python and Java.
• DevOps Infrastructure: Experience with infrastructure-as-code (IaC) tools such as CloudFormation and managing CI/CD deployment pipelines.
• Analytical Problem-Solving Skills: Exceptional analytical mindset with a systematic approach to debugging complex, distributed, asynchronous systems under pressure.
• Communication Skills: Excellent verbal and written communication abilities, with a proven track record of translating complex technical roadblocks into actionable solutions for cross-functional teams and stakeholders.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.