Kafka Flink Engineer

NSKH Tech; India; Remote; INR 3,600,000 - 6,000,000; Full time

Job summary

NSKH Tech is seeking a Confluent Flink Engineer to design, develop, and optimize real-time streaming applications using Apache Flink, Confluent Kafka, and Kafka Streams. You will build large-scale streaming ETL pipelines and implement event-driven architectures in a cloud environment.

The role requires 6+ years of software/data engineering experience, strong streaming knowledge, and proficiency in Java/Scala/Python. Remote work with IST night shifts and cross-functional collaboration.

Qualifications

• 6+ years of software or data engineering experience.
• Hands-on with Apache Flink.
• Extensive experience with Confluent Kafka and Kafka Streams.
• Strong understanding of real-time streaming and event-driven architectures.
• Experience with distributed systems and large-scale data processing.
• Proficiency in Java, Scala, or Python.
• Experience building streaming ETL pipelines.
• Knowledge of Kafka Connect, Schema Registry, Confluent Platform Components.
• Knowledge of data partitioning, state management, checkpointing, fault tolerance.
• Experience with cloud platforms (AWS/Azure/GCP).
• Familiarity with CI/CD and DevOps practices.

Responsibilities

• Design, develop, and maintain real-time data processing applications using Apache Flink.
• Build and manage large-scale streaming ETL pipelines.
• Develop event-driven architectures using Confluent Kafka and Kafka Streams.
• Implement real-time data transformation, aggregation, enrichment, and analytics workflows.

Skills

Apache Flink

Confluent Kafka

Kafka Streams

Real-time streaming

Distributed systems

Java/Scala/Python

Streaming ETL pipelines

Cloud platforms (AWS/Azure/GCP)

DevOps / CI/CD

Kubernetes

Tools

Kafka Connect

Schema Registry

Confluent Platform

Prometheus

Grafana

Datadog

Job description

Job Title: Confluent Flink Engineer

Experience: 6+ Years

Location: Remote

Shift Timings: 2:00 PM – 11:00 PM IST

Role Overview

We are seeking an experienced Confluent Flink Engineer with strong expertise in real-time data streaming, event-driven architectures, and large-scale distributed systems. The ideal candidate should have hands‑on experience with Apache Flink, Confluent Kafka, Kafka Streams, and streaming ETL pipelines, along with a strong understanding of cloud‑native data processing solutions.

The role involves designing, developing, and optimizing real‑time streaming applications that process high-volume data efficiently while ensuring scalability, fault tolerance, and performance.

Key ResponsibilitiesStream Processing & Data Engineering

• Design, develop, and maintain real-time data processing applications using Apache Flink.
• Build and manage large-scale streaming ETL pipelines.
• Develop event-driven architectures using Confluent Kafka and Kafka Streams.
• Implement real-time data transformation, aggregation, enrichment, and analytics workflows.

Kafka & Confluent Platform

• Configure and manage Confluent Kafka environments.
• Work with Kafka Connect, Schema Registry, and other Confluent Platform components.
• Design efficient topic structures, partitioning strategies, and consumer group architectures.
• Ensure reliable event processing and message delivery.

Performance & Scalability

• Optimize Flink jobs for performance, throughput, and resource utilization.
• Implement checkpointing, state management, and fault-tolerant processing strategies.
• Monitor and troubleshoot streaming applications and data pipelines.
• Ensure scalability and high availability of streaming platforms.

Cloud & DevOps

• Deploy and manage streaming workloads on AWS, Azure, or GCP.
• Participate in CI/CD pipeline implementation and automation.
• Collaborate with DevOps teams to improve deployment, monitoring, and operational practices.

Collaboration

• Work closely with Data Engineers, Architects, Product Teams, and Business Stakeholders.
• Participate in architecture discussions and technical design reviews.
• Contribute to best practices, documentation, and knowledge sharing.

Required Skills & Experience

• 6+ years of software engineering or data engineering experience.
• Strong hands‑on experience with Apache Flink.
• Extensive experience with Confluent Kafka and Kafka Streams.
• Strong understanding of real-time streaming and event-driven architectures.
• Experience with distributed systems and large-scale data processing.
• Proficiency in Java, Scala, or Python.
• Experience building and managing streaming ETL pipelines.
• Strong knowledge of:
  • Kafka Connect
  • Schema Registry
  • Confluent Platform Components
• Strong understanding of:
  • Data Partitioning
  • State Management
  • Checkpointing
  • Fault Tolerance
• Experience with AWS, Azure, or GCP cloud platforms.
• Familiarity with CI/CD pipelines and DevOps practices.

Preferred Skills

• Experience with Kubernetes and containerized deployments.
• Knowledge of data lake and modern data platform architectures.
• Exposure to monitoring tools such as Prometheus, Grafana, or Datadog.
• Experience with Spark Streaming or other stream processing technologies.

Key Competencies

• Real-Time Data Streaming
• Apache Flink Development
• Confluent Kafka & Kafka Streams
• Distributed Systems
• Event-Driven Architecture
• Streaming ETL Pipelines
• Cloud Data Engineering
• Performance Optimization
• CI/CD & DevOps

Skills: kafka,data,processing,etl,devops,cloud,pipelines,flink