Senior Event Streaming Engineer (Kafka / Azure Event Hubs)

Apptoza Inc.; Vancouver; Remote; CAD 140,000 - 200,000; Full time

Job summary

Apptoza Inc. is seeking a hands-on Senior Engineer/Lead to design and implement an event-driven integration capability using Azure Event Hubs and Kafka-compatible interfaces.

You will own the integration design, Event Hubs configuration, security, and downstream consumer enablement, collaborating with Cloud Engineering and Information Security teams, while infrastructure provisioning is handled by Cloud Engineering.

Qualifications

• Hands-on experience designing and implementing Azure Event Hubs and Apache Kafka solutions.
• Strong understanding of Kafka topics, partitions, consumer groups, offsets and delivery semantics.
• Experience with event-driven architecture, partitioning, schema compatibility.
• Practical knowledge of Azure security, including Entra ID, RBAC, encryption, credential management.
• Experience with retries, duplicate handling, recovery and replay in event streams.
• Ability to design, implement, troubleshoot and test end-to-end integration solutions.

Responsibilities

• Design the integration approach, including direct publishing versus a Kafka bridge.
• Define and configure event hubs, partitions, consumer groups, retention and throughput.
• Implement authentication, RBAC, encryption and secure credential management.
• Implement event ingestion, topic mapping, serialization/deserialization and schema compatibility.
• Enable downstream consumers using Kafka protocol or Azure-native SDKs, including independent consumer groups and checkpoint management.
• Establish monitoring, alerting, operational runbooks, and technical documentation.

Skills

Azure Event Hubs

Apache Kafka

Kafka producers/consumers

Event-driven architecture

Schema compatibility

Azure security

RBAC

Retries and replay

End-to-end integration

Tools

Terraform

Bicep

Azure Monitor

Job description

Location: Remote

Contract: Long term

Role Description

We are seeking a hands‑on Senior Engineer / Lead to design and implement an event‑driven integration capability using Azure Event Hubs and Kafka‑compatible interfaces.

This role would own the integration design, Event Hubs configuration, security and access controls, event ingestion, and downstream consumer enablement. The role would collaborate closely with Cloud Engineering, Information Security, and application teams, with Azure infrastructure provisioning and network configuration handled by the Cloud Engineering team.

Responsibilities

Requirements and Design

• Assess upstream event producer capabilities, Kafka endpoints, topics, schemas, sample payloads, throughput, latency, and ordering requirements.
• Design the integration approach, including direct publishing versus a Kafka bridge, partitioning strategy, availability, and disaster recovery.

Event Hubs Configuration and Security

• Define and configure event hubs, partitions, consumer groups, retention, and throughput settings; coordinate infrastructure provisioning with Cloud Engineering.
• Implement appropriate authentication, RBAC, least‑privilege access, encryption controls, and secure credential management.

Event Ingestion and Consumer Enablement

• Implement event ingestion, topic mapping, serialization/deserialization, schema compatibility, and event metadata handling.
• Enable downstream consumers using Kafka protocol or Azure‑native SDKs, including independent consumer groups and checkpoint management.

Reliability, Testing, and Operations

• Implement retry, recovery, duplicate handling, and event replay strategies; validate end‑to‑end event delivery.
• Establish monitoring, alerting, operational runbooks, and technical documentation for the foundational implementation.

Qualifications

A. Mandatory Qualifications

• Hands‑on experience designing and implementing solutions using Azure Event Hubs and Apache Kafka.
• Strong understanding of Kafka producers/consumers, topics, partitions, partition keys, consumer groups, offsets, and delivery semantics.
• Experience with event‑driven architecture, partitioning strategies, serialization/deserialization, and schema compatibility.
• Practical knowledge of Azure security, including Microsoft Entra ID, RBAC, authentication, authorization, encryption, and secure credential management.
• Experience implementing reliable event processing, including retries, duplicate handling, recovery, and replay.
• Ability to independently design, implement, troubleshoot, and test end‑to‑end integration solutions.

Preferred Qualifications

• Programming proficiency in C#/.NET for Kafka integration or bridge development.
• Experience with JSON, Avro, schema registries, and event schema versioning.
• Familiarity with Azure Monitor, Log Analytics, Terraform/Bicep, and CI/CD automation.
• Knowledge of high availability, disaster recovery, and production support; experience in financial services or other regulated environments.

Relevant Azure or Kafka certifications, particularly:

• Confluent Certified Developer for Apache Kafka (CCDAK)