Senior Confluent Kafka Engineer - Real-Time Cloud Streaming

BCforward; Minneapolis (MN); On-site; USD 90,000 - 103,000; Full time

Job summary

BCforward seeks a Sr. Confluent Kafka Developer to join our team in Minneapolis, MN. You will design, build, and operate scalable streaming pipelines using Confluent Kafka and Apache Flink for high-performance applications.

The role requires 3+ years in distributed systems, 2+ years in Python for Kafka apps, and hands-on experience with Schema Registry, Terraform, and AWS services. You will collaborate with data engineering to ensure data quality and reliability.

Qualifications

• 3 years in software development focused on Apache Kafka and distributed systems.
• 2 years of Python experience for Kafka applications.
• Hands-on Confluent Kafka with Schema Registry.
• Infrastructure as Code with Terraform.
• Experience with AWS services such as EC2, S3, IAM, and Lambda.
• Knowledge of data integration, ETL, and pipeline orchestration.
• Operational monitoring and performance tuning of streaming pipelines.
• Strong troubleshooting skills across Kafka and its ecosystem.
• Analytical problem-solving, attention to detail, and clear communication.
• Proven collaboration in cross-functional team environments.

Responsibilities

• Develop, implement, and manage streaming data pipelines using Confluent Kafka and Apache Flink.
• Integrate streaming solutions with AWS and Snowflake platforms in collaboration with data engineering teams.
• Design, build, and maintain real-time pipelines with a focus on data quality, integrity, and reliability.
• Use Terraform to manage and automate infrastructure deployments on AWS or other clouds.
• Develop and maintain CI/CD pipelines for Kafka clusters and related services.
• Collaborate with development, DevOps, and data science teams to deliver optimal data solutions.
• Provide support and guidance on Kafka issues, patterns, and best practices.
• Evaluate new Kafka and streaming technologies and recommend improvements.

Skills

Apache Kafka

Python

Schema Registry

Data integration

ETL

Pipeline orchestration

Monitoring

Troubleshooting

Cross-functional collaboration

Communication

Tools

Terraform

AWS

Docker

Kubernetes

Kafka Connect

ksqlDB

Snowflake

Job description

BCforward seeks a Sr. Confluent Kafka Developer to join our team in Minneapolis, MN. You will design, build, and operate scalable streaming pipelines using Confluent Kafka and Apache Flink for high-performance applications.

The role requires 3+ years in distributed systems, 2+ years in Python for Kafka apps, and hands-on experience with Schema Registry, Terraform, and AWS services. You will collaborate with data engineering to ensure data quality and reliability.