Senior Data Streaming Engineer

Unify Technologies; Woodlawn (MD); On-site; USD 124,000 - 220,000; Full time

Job summary

Unify Technologies Ltd invites a Senior Data Streaming Engineer with deep Kafka and Flink expertise to join our on-site team in Woodlawn, MD. The role leads enterprise streaming architecture, builds reusable components, and raises security standards across CI/CD and deployment.

You will mentor teams, define reference architectures, and drive secure, scalable streaming solutions while ensuring compliance with federal security controls and stateful processing requirements.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10+ years of progressive IT experience, including senior engineering, technical leadership, or advisory responsibilities.
• 5+ years of experience in enterprise-scale data streaming or secure software delivery environments.
• Expert-level knowledge of Apache Kafka, Apache Flink, Java, Avro, and Confluent Schema Registry.
• Strong understanding of Kafka architecture, including brokers, partitions, replication, consumer groups, ISR behavior, retention, ACLs, and cluster operations.
• Experience designing and implementing secure CI/CD and DevSecOps pipelines using Jenkins, Maven, Bitbucket, and Nexus.
• Hands-on experience with vulnerability management, secure coding, dependency governance, and POA&M-aligned remediation.
• Experience implementing secure Kafka client authentication and communication using SSL/mTLS and SASL/OAUTHBEARER.
• Knowledge of schema governance, schema evolution, stateful streaming applications, and fault-tolerant distributed systems.
• Demonstrated ability to develop enterprise documentation, reference architectures, reusable components, and engineering standards.
• Strong communication, problem-solving, mentoring, and cross-functional collaboration skills.
• Ability to obtain and maintain SSA Public Trust clearance.

Responsibilities

• Define and maintain enterprise reference architectures, standards, and best practices for Kafka, Flink, Kafka Streams, and microservices.
• Develop reusable streaming libraries, starter templates, and reference implementations to support consistent adoption across teams.
• Provide hands-on technical guidance to application and platform teams on streaming architecture and implementation.
• Serve as a senior escalation point for complex streaming issues, including consumer lag, serialization failures, state management, rebalancing, and performance bottlenecks.
• Establish standards for schema governance, fault tolerance, scalability, and distributed data processing.
• Collaborate with DevOps and Platform Engineering teams to establish secure CI/CD pipelines and deployment standards.
• Develop reusable pipeline patterns incorporating vulnerability scanning, dependency management, artifact promotion, and remediation workflows.
• Implement automated schema validation and Kafka topic provisioning checks within CI/CD pipelines.
• Define strategies for zero-downtime deployments, schema evolution, and safe stateful application upgrades.
• Integrate security policies and compliance requirements into software delivery processes.
• Define secure coding and dependency management standards for Java, Kafka, and Flink applications.
• Analyze vulnerability scan results and guide engineering teams in remediation.
• Support compliance with NIST security controls, POA&M, and federal patch management requirements.
• Implement secure Kafka client configurations using SSL/mTLS and SASL/OAUTHBEARER.
• Promote secure microservice development and enterprise security best practices.
• Apply deep knowledge of Kafka brokers, partitions, replication, consumer groups, ACLs, RBAC, retention policies, and cluster health.
• Troubleshoot ISR fluctuations, replication delays, consumer lag, and cluster performance issues.
• Guide teams through production incidents involving distributed streaming applications.
• Establish operational standards for monitoring, reliability, scalability, and fault tolerance.

Skills

Leadership

Mentoring

Cross-functional collaboration

Problem solving

Communication

Clearance eligibility

Education

Bachelor's degree in CS/IT or related field

Tools

Apache Kafka

Apache Flink

Java

Avro

Confluent Schema Registry

Jenkins

Maven

Bitbucket

Nexus

Docker

Kubernetes

OpenShift

Job description

We have an exciting opportunity for a Senior Data Streaming Engineer - Kafka & Flink in Woodlawn, MD. This is a long-term contract position (12+ months), with an onsite requirement of 5 days a week.

Employment Type: Permanent

Work Arrangement: Onsite, 5 days a week

Clearance: Must be eligible to obtain and maintain SSA Public Trust clearance

Position Overview

Senior Data Streaming Engineer with expertise in Apache Kafka and Apache Flink to join an enterprise Center of Excellence (CoE) focused on mission-critical, real-time data streaming and secure software delivery.

The ideal candidate will provide technical leadership across application and platform teams, establish enterprise streaming standards, design secure and scalable streaming architectures, and drive DevSecOps best practices. This role requires deep hands-on expertise in Kafka, Flink, Java, schema governance, CI/CD security, and distributed systems troubleshooting.

Note: This is an onsite position requiring attendance five days per week in Woodlawn, MD.

Key ResponsibilitiesEnterprise Streaming Architecture & Technical Leadership

• Define and maintain enterprise reference architectures, standards, and best practices for Kafka, Flink, Kafka Streams, and microservices.
• Develop reusable streaming libraries, starter templates, and reference implementations to support consistent adoption across teams.
• Provide hands-on technical guidance to application and platform teams on streaming architecture and implementation.
• Serve as a senior escalation point for complex streaming issues, including consumer lag, serialization failures, state management, rebalancing, and performance bottlenecks.
• Establish standards for schema governance, fault tolerance, scalability, and distributed data processing.

CI/CD & DevSecOps

• Collaborate with DevOps and Platform Engineering teams to establish secure CI/CD pipelines and deployment standards.
• Develop reusable pipeline patterns incorporating vulnerability scanning, dependency management, artifact promotion, and remediation workflows.
• Implement automated schema validation and Kafka topic provisioning checks within CI/CD pipelines.
• Define strategies for zero-downtime deployments, schema evolution, and safe stateful application upgrades.
• Integrate security policies and compliance requirements into software delivery processes.

Security & Vulnerability Management

• Define secure coding and dependency management standards for Java, Kafka, and Flink applications.
• Analyze vulnerability scan results and guide engineering teams in remediation.
• Support compliance with NIST security controls, Plans of Action and Milestones (POA&Ms), and federal patch management requirements.
• Implement secure Kafka client configurations using SSL/mTLS and SASL/OAUTHBEARER.
• Promote secure microservice development and enterprise security best practices.

Kafka Operations & Troubleshooting

• Apply deep knowledge of Kafka brokers, partitions, replication, consumer groups, ACLs, RBAC, retention policies, and cluster health.
• Troubleshoot in-sync replica (ISR) fluctuations, replication delays, consumer lag, and cluster performance issues.
• Guide teams through production incidents involving distributed streaming applications.
• Establish operational standards for monitoring, reliability, scalability, and fault tolerance.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10+ years of progressive IT experience, including senior engineering, technical leadership, or advisory responsibilities.
• 5+ years of experience in enterprise-scale data streaming or secure software delivery environments.
• Expert-level knowledge of Apache Kafka, Apache Flink, Java, Avro, and Confluent Schema Registry.
• Strong understanding of Kafka architecture, including brokers, partitions, replication, consumer groups, ISR behavior, retention, ACLs, and cluster operations.
• Experience designing and implementing secure CI/CD and DevSecOps pipelines using Jenkins, Maven, Bitbucket, and Nexus.
• Hands-on experience with vulnerability management, secure coding, dependency governance, and POA&M-aligned remediation.
• Experience implementing secure Kafka client authentication and communication using SSL/mTLS and SASL/OAUTHBEARER.
• Knowledge of schema governance, schema evolution, stateful streaming applications, and fault-tolerant distributed systems.
• Demonstrated ability to develop enterprise documentation, reference architectures, reusable components, and engineering standards.
• Strong communication, problem-solving, mentoring, and cross-functional collaboration skills.
• Ability to obtain and maintain SSA Public Trust clearance.

Preferred Qualifications

• Experience working within a Center of Excellence, architecture council, or enterprise technology governance team.
• Previous experience supporting federal government agencies, such as SSA, CMS, DHS, or DOJ.
• Experience with Docker, Kubernetes, or Red Hat OpenShift and secure containerized application deployments.
• Familiarity with security and code quality tools such as Black Duck, SonarQube, and Fortify.
• Experience with Confluent Platform RBAC and enterprise Kafka security controls.
• Experience designing large-scale streaming solutions and defining standards for partitioning, schema evolution, state management, and fault tolerance.
• Experience building reusable libraries, starter templates, and shared components for streaming and microservices.
• Advanced distributed systems troubleshooting and production incident management experience.
• Experience with Kafka Streams or similar stateful streaming frameworks.