Software Engineer: IV (Lead)

Dale Workforce Solutions; Saint Paul (MN); On-site; USD 120,000 - 160,000; Full time

Job summary

Dale Workforce Solutions is seeking a software engineer to design and manage real-time streaming pipelines using Apache Kafka and Apache Flink. You will integrate diverse data sources with AWS and Snowflake, ensure data quality, and drive reliability across platforms.

The role emphasizes automation with Terraform and CI/CD, performance tuning of pipelines, and providing hands-on Kafka guidance to teams. Ideal candidates have 3+ years in Kafka and distributed systems and 2+ years in Python.

Qualifications

• Requires at least three years of software development experience focused on Apache Kafka and distributed systems.
• Includes at least two years of Python experience.
• Hands-on experience with Confluent Kafka, Schema Registry, Terraform, and AWS.

Responsibilities

• Develop and manage real-time streaming pipelines using Apache Kafka and Apache Flink.
• Integrate data sources with AWS and Snowflake while maintaining data quality.
• Automate infrastructure and deployments with Terraform and CI/CD.

Skills

Apache Kafka

Confluent Cloud

Python

Distributed Systems

Apache Flink

Terraform

AWS

Schema Registry

Data Integration

ETL

Data Pipeline Orchestration

CI/CD

Operational Monitoring

Snowflake

Java

Kafka Connect

Job description

Develop and manage real-time streaming pipelines using Confluent Kafka and Apache Flink, integrating data sources with platforms such as AWS and Snowflake while maintaining data quality and reliability. Automate infrastructure and deployments with Terraform and CI/CD, optimize pipeline performance, and support teams with Kafka guidance and troubleshooting.

Requirements

Requires at least three years of software development experience focused on Apache Kafka and distributed systems, including at least two years of Python experience, plus hands‑on experience with Confluent Kafka, Schema Registry, Terraform, and AWS. Preferred qualifications include Confluent Cloud, Apache Flink, ksqlDB, Kafka Connect, Snowflake, Java, cloud‑native DevOps tools, and relevant certifications.

Key Skills

• Apache Kafka
• Confluent Cloud
• Python
• Distributed Systems
• Apache Flink
• Terraform
• AWS
• Schema Registry
• Data Integration
• ETL
• Data Pipeline Orchestration
• CI/CD
• Operational Monitoring
• Snowflake
• Java
• Kafka Connect