Real-Time Kafka Engineer – Confluent Platform

Fractal; United States; Remote; USD 120,000 - 190,000; Full time

Job summary

Fractal is seeking a skilled Kafka Engineer to design, build, and maintain high-performance real-time streaming pipelines using Confluent Platform. The ideal candidate will deploy and scale Kafka clusters across on-prem and cloud environments, ensuring high availability.

You will implement Kafka Connect, Schema Registry, KSQL, and FlinkSQL, and integrate pipelines with data lakes and warehouses. Collaboration with data engineers and backend teams is essential for success.

Qualifications

• Experience designing and operating real-time streaming pipelines.
• Hands-on with Kafka ecosystem components and Confluent Platform.
• Ability to deploy on-prem and cloud Kafka clusters with monitoring.

Responsibilities

• Design, build, and manage Kafka and Confluent infrastructure for high throughput streaming applications.
• Implement Kafka Connect, Schema Registry, and KSQL (KSQLDB), FlinkSQL
• Ensure high availability, performance tuning, and monitoring of Kafka clusters
• Collaborate with data engineers and backend teams to integrate streaming pipelines with data lakes, warehouses.
• Set up and enforce Kafka security mechanisms (SASL, SSL, ACLs, RBAC)
• Create and manage CI/CD pipelines for Kafka components using tools like Git, Jenkins, etc.
• Monitor and troubleshoot issues using tools like Confluent Control Center, DataDog

Skills

Java

Python

Stream processing

Event-driven architecture

Tools

Apache Kafka

Confluent Platform

Kafka Connect

Schema Registry

KSQL

FlinkSQL

Git

Jenkins

Kubernetes

Helm

Confluent Control Center

DataDog

Cloud platforms (AWS/Azure/GCP)

Job description

Fractal is seeking a skilled Kafka Engineer to design, build, and maintain high-performance real-time streaming pipelines using Confluent Platform. The ideal candidate will deploy and scale Kafka clusters across on-prem and cloud environments, ensuring high availability.

You will implement Kafka Connect, Schema Registry, KSQL, and FlinkSQL, and integrate pipelines with data lakes and warehouses. Collaboration with data engineers and backend teams is essential for success.