Senior Confluent Kafka Flink Streaming Architect Onsite in Columbus OH 6 Months

Genius Business Solutions; Columbus (OH); On-site; USD 150,000 - 210,000; Full time

Job summary

Genius Business Solutions Inc. (GBSI) is seeking a Senior Streaming Architect/Engineer to design, build, and operate our real-time data platform on Confluent Cloud and Apache Flink. You will guide cloud-native, event-driven architecture on AWS and GCP, set technical direction, and mentor engineers while interfacing with business stakeholders.

You will lead migrations from legacy systems, implement fault-tolerance, and drive observability and security across global environments.

Qualifications

• 12+ years in software, integration, or data engineering, with 5+ years hands-on Kafka/Confluent in production.
• Strong hands-on experience with Apache Flink (DataStream API and Flink SQL).
• Deep Kafka knowledge: partitioning, replication, retention, Kafka Connect, Kafka Streams, ksqlDB.
• Experience running Confluent Cloud in enterprise settings on AWS and/or GCP.

Responsibilities

• Architect multi-region, highly available Confluent Cloud environments with DR strategies (Cluster Linking, cross-region replication) for 200K+ events/sec.
• Design and build real-time pipelines in Apache Flink (DataStream, Table, and SQL APIs).
• Implement stateful, fault-tolerant processing with checkpointing, savepoints, and exactly-once semantics; optimize performance.
• Build and manage Kafka Connect source/sink connectors with SMTs and DLQ/retry patterns.
• Own schema governance through Schema Registry and compatibility rules.
• Develop event-driven microservices and Kafka Streams / ksqlDB apps using Java and Spring Boot.
• Lead migrations of on-prem workloads to Confluent Cloud and AWS with zero-downtime cutover.
• Automate infrastructure with Terraform and GitOps CI/CD (Harness, Jenkins).
• Set up secure connectivity (Private Service Connect, VPC, PrivateLink), RBAC, SSL/SASL, and audit logging.
• Lead observability (Prometheus, Grafana, CloudWatch, Splunk); drive incident response and RCA.

Skills

Confluent Kafka

Apache Flink

Kafka Streams

ksqlDB

Java

Spring Boot

Python

SQL

Terraform

Docker

Kubernetes

CI/CD

AWS

GCP

Tools

Confluent Cloud

Terraform

Docker

Kubernetes

Jenkins

Harness

Prometheus

Grafana

Splunk

Job description

About Genius Business Solutions Inc. (GBSI)

Featured in CNBC, Digital Journal, Fox News, and CIO Review, GBSI is a globally recognized IT services leader with 20+ years serving Fortune 500 organizations. Our teams deliver solutions across automotive, manufacturing, retail, and pharmaceutical industries. Headquartered in Moline, IL, GBSI supports clients and consultants throughout the US, Canada, Europe, and India.

Join our innovative team to lead enterprise-grade SAP solutions and drive digital transformation initiatives across global enterprises.

Job Summary

We are looking for a Senior Streaming Architect / Engineer to design, build, and run our enterprise real-time data platform on Confluent Cloud and Apache Flink. You will lead the move from legacy batch and ETL workloads to a cloud-native, event-driven architecture on AWS and GCP. You will set technical direction, mentor engineers, and work with business and executive stakeholders.

Key Responsibilities

• Architect multi-region, highly available Confluent Cloud environments with DR strategies (Cluster Linking, cross-region replication) supporting 200K+ events/sec.
• Design and build real-time pipelines in Apache Flink (DataStream, Table, and SQL APIs): event-time processing, watermarks, late-event handling, windowed aggregations (tumbling, hopping, session), and stream joins.
• Implement stateful, fault-tolerant processing with checkpointing, savepoints, and exactly-once semantics; tune parallelism, state backends, and operator chaining.
• Build and manage Kafka Connect source/sink connectors (CDC, RDBMS, S3, analytics platforms) with SMTs and DLQ/retry patterns.
• Own schema governance through Schema Registry (Avro/JSON/Protobuf) and compatibility rules.
• Develop event-driven microservices and Kafka Streams / ksqlDB applications using Java and Spring Boot.
• Lead migrations of on-prem middleware and Confluent Platform workloads to Confluent Cloud and AWS (EKS) with zero-downtime cutover.
• Automate infrastructure with Terraform (Confluent, AWS, GCP providers) and GitOps CI/CD (Harness, Jenkins).
• Implement secure connectivity (Private Service Connect, VPC, PrivateLink), RBAC, ACLs, SSL/SASL, and audit logging.
• Set up observability (Prometheus, Grafana, CloudWatch, Splunk); lead incident response, RCA, and performance tuning.
• Optimize partitioning, retention, and capacity to reduce infrastructure cost.
• Lead and mentor Kafka/Flink engineers, run training sessions, and communicate architecture trade-offs to leadership.

Required Qualifications

• 12+ years in software, integration, or data engineering, with 5+ years hands‑on with Apache Kafka / Confluent in production. (Overall 16 years required)
• Strong hands‑on experience with Apache Flink (DataStream API and Flink SQL), including state management and checkpoint tuning.
• Deep Kafka knowledge: partitioning, replication/ISR, retention, compaction, Kafka Connect, Kafka Streams, ksqlDB.
• Experience running Confluent Cloud in an enterprise setting on AWS and/or GCP.
• Advanced Java (11+) and Spring Boot; proficiency in Python and SQL.
• Experience with Terraform, Docker, and Kubernetes (EKS / GKE / OpenShift).
• Experience with CI/CD tools (Jenkins, Harness) and monitoring stacks (Prometheus, Grafana, Splunk).
• Proven cloud migration experience, from assessment and planning through cutover.
• Strong communication skills and experience with stakeholders in regulated industries.

Preferred Qualifications

• Confluent Certified Developer / Administrator for Apache Kafka; Confluent Certified Data Streaming Engineer (Flink).
• AWS Solutions Architect certification.
• Experience with Amazon MSK, Managed Service for Apache Flink, Kinesis, EMR, Glue, and MWAA.
• PySpark experience for batch and streaming ETL.
• Familiarity with Snowflake, BigQuery, PostgreSQL, Oracle, and DB2 as sources or sinks.
• Experience with CDC patterns and legacy integration platforms (IBM MQ, IIB, mainframe).
• Domain background in healthcare, insurance, banking/finance, or retail.

Tools & Technologies

Confluent Cloud, Apache Kafka, Apache Flink, ksqlDB, Kafka Streams, Kafka Connect, Schema Registry, AWS (EKS, MSK, S3, IAM, VPC), GCP (GKE, Pub/Sub, PSC), Terraform, Kubernetes, Docker, Jenkins, Harness, Java / Spring Boot, Python, PySpark, Prometheus, Grafana, Splunk.

Must-have

• Confluent Kafka production experience and hands‑on Apache Flink (not just Kafka Streams).
• Ask about: Flink checkpointing/savepoints, watermarks and late events, exactly‑once guarantees, Cluster Linking DR design.
• Strong signal: Confluent Cloud migration from on‑prem, Terraform‑based provisioning, and Confluent certifications.
• Profile titles to search: Kafka Architect, Confluent Architect, Streaming Data Engineer, Flink Engineer, Event Streaming Solution Architect.

Equal Employment Opportunity

GeniusBSI is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.

All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status, or any other basis as protected by federal, state, or local law.