Kafka Architect/Engineer

Themesoft Inc; Columbus (OH); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive compensation

Career growth opportunities

Tuition reimbursement

Job summary

The Themesoft Inc team is seeking a Senior Streaming Architect/Engineer to design and run a real-time data platform on Confluent Cloud and Apache Flink in Columbus, OH. You will lead cloud-native, event-driven transformations and mentor engineers while engaging with business stakeholders.

Responsibilities include building multi-region, highly available Confluent Cloud environments, implementing stateful processing, and driving migration of on-prem workloads to cloud.

Qualifications

• 12+ years software, integration, or data engineering experience with 5+ years in production Kafka/Confluent.
• Deep hands-on Flink experience including state management and checkpoint tuning.
• Proven cloud migration, multi-region Confluent Cloud, and DR design in AWS/GCP.
• Strong Java/Spring Boot plus Python and SQL proficiency.
• Experience with Terraform, Docker, Kubernetes, and CI/CD tooling.

Responsibilities

• Architect multi-region Confluent Cloud environments with DR, handling 200K+ events/sec.
• Build real-time pipelines in Flink with event-time processing and windowed aggregations.
• Implement exactly-once processing, stateful streaming, and performance tuning.
• Develop Kafka Connect pipelines with CDC, S3, and analytics sinks; ensure schema governance.
• Lead migrations of on-prem workloads to Confluent Cloud and cloud-native platforms.
• Establish observability, incident response, and cost optimization strategies.

Skills

Apache Kafka

Confluent Cloud

Apache Flink

Java / Spring Boot

Python

SQL

Terraform

Docker

Kubernetes

CI/CD

Tools

AWS

GCP

Jenkins

Harness

EKS/GKE

Schema Registry

Kafka Connect

Job description

Job Title: Senior Confluent Kafka Architect/Engineer

Location: Columbus, OH

Experience: 12+ years overall (8+ years for Senior Engineer level); 5+ years in event streaming

About the Role

We are looking for a Senior Streaming Architect / Engineer to design, build, and run our enterprise real-time data platform on Confluent Cloud and Apache Flink. You will lead the move from legacy batch and ETL workloads to a cloud-native, event-driven architecture on AWS and GCP. You will set technical direction, mentor engineers, and work with business and executive stakeholders.

Key Responsibilities

• Architect multi-region, highly available Confluent Cloud environments with DR strategies (Cluster Linking, cross-region replication) supporting 200K+ events/sec.
• Design and b*9 uild real-time pipelines in Apache Flink (DataStream, Table, and SQL APIs): event-time processing, watermarks, late-event handling, windowed aggregations (tumbling, hopping, session), and stream joins.
• Implement stateful, fault-tolerant processing with checkpointing, savepoints, and exactly-once semantics; tune parallelism, state backends, and operator chaining.
• Build and manage Kafka Connect source/sink connectors (CDC, RDBMS, S3, analytics platforms) with SMTs and DLQ/retry patterns.
• Own schema governance through Schema Registry (Avro/JSON/Protobuf) and compatibility rules.
• Develop event-driven microservices and Kafka Streams / ksqlDB applications using Java and Spring Boot.
• Lead migrations of on-prem middleware and Confluent Platform workloads to Confluent Cloud and AWS (EKS) with zero-downtime cutover.
• Automate infrastructure with Terraform (Confluent, AWS, GCP providers) and GitOps CI/CD (Harness, Jenkins).
• Implement secure connectivity (Private Service Connect, VPC, PrivateLink), RBAC, ACLs, SSL/SASL, and audit logging.
• Set up observability (Prometheus, Grafana, CloudWatch, Splunk); lead incident response, RCA, and performance tuning.
• Optimize partitioning, retention, and capacity to reduce infrastructure cost.
• Lead and mentor Kafka/Flink engineers, run training sessions, and communicate architecture trade-offs to leadership.

Required Qualifications

• 12+ years in software, integration, or data engineering, with 5+ years hands-on with Apache Kafka / Confluent in production.
• Strong hands-on experience with Apache Flink (DataStream API and Flink SQL), including state management and checkpoint tuning.
• Deep Kafka knowledge: partitioning, replication/ISR, retention, compaction, Kafka Connect, Kafka Streams, ksqlDB.
• Experience running Confluent Cloud in an enterprise setting on AWS and/or GCP.
• Advanced Java (11+) and Spring Boot; proficiency in Python and SQL.
• Experience with Terraform, Docker, and Kubernetes (EKS / GKE / OpenShift).
• Experience with CI/CD tools (Jenkins, Harness) and monitoring stacks (Prometheus, Grafana, Splunk).
• Proven cloud migration experience, from assessment and planning through cutover.
• Strong communication skills and experience with stakeholders in regulated industries.

Preferred Qualifications

• Confluent Certified Developer / Administrator for Apache Kafka; Confluent Certified Data Streaming Engineer (Flink).
• Experience with Amazon MSK, Managed Service for Apache Flink, Kinesis, EMR, Glue, and MWAA.
• PySpark experience for batch and streaming ETL.
• Familiarity with Snowflake, BigQuery, PostgreSQL, Oracle, and DB2 as sources or sinks.
• Experience with CDC patterns and legacy integration platforms (IBM MQ, IIB, mainframe).
• Domain background in healthcare, insurance, banking/finance, or retail.

Tools & Technologies

Confluent Cloud, Apache Kafka, Apache Flink, ksqlDB, Kafka Streams, Kafka Connect, Schema Registry, AWS (EKS, MSK, S3, IAM, VPC), GCP (GKE, Pub/Sub, PSC), Terraform, Kubernetes, Docker, Jenkins, Harness, Java / Spring Boot, Python, PySpark, Prometheus, Grafana, Splunk.

Must-have

• Confluent Kafka production experience and hands-on Apache Flink (not just Kafka Streams).
• Ask about: Flink checkpointing/savepoints, watermarks and late events, exactly-once guarantees, Cluster Linking DR design.
• Strong signal: Confluent Cloud migration from on-prem, Terraform-based provisioning, and Confluent certifications.
• Profile titles to search: Kafka Architect, Confluent Architect, Streaming Data Engineer, Flink Engineer, Event Streaming Solution Architect.