Kafka Senior Developer

Zone IT Solutions Pty Ltd; Melbourne; Vor Ort; AUD 120.000 - 180.000; Vollzeit

Zusammenfassung

Zone IT Solutions Pty Ltd in Melbourne is seeking an experienced Integration Engineer to design and implement data-driven, event-based integration patterns.

You will develop Java/Spring Boot connectors, manage Kafka pipelines on Confluent Cloud/NGIP, and build Flink SQL streaming jobs for transformation, validation, and routing. Strong knowledge of Schema Registry and API integrations via Kong Gateway is required.

Qualifikationen

• Must develop custom integration code using Java / Spring Boot for source system connectivity.
• Must build and maintain Kafka producers/consumers and event-driven services on Confluent Cloud/NGIP.
• Must develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
• Must implement integration with Schema Registry, topic schemas, and message validation controls; support HTTP/API-based integrations via Kong Gateway.
• Must configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling.

Aufgaben

• Develop custom integration code using Java / Spring Boot for source system connectivity where managed connectors are not applicable.
• Build and maintain Kafka producers / consumers, topic integration logic, and event-driven services on Confluent Cloud / NGIP.
• Develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
• Implement integration with Schema Registry, topic schemas, and message validation controls.
• Support HTTP/API-based integrations through Kong Gateway for NGIP to target application communication
• Configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling.
• Perform unit / component testing for custom code, Flink jobs, and interface logic.
• Support SIT, pipeline testing, defect fixing, replay, and root-cause analysis during test execution
• Support E2E testing with downstream teams and resolve integration issues across connectors, transformations, APIs, and Kafka topics
• Work with GitLab CI/CD pipelines for build, deployment, regression automation, and release support

Kenntnisse

Java

Spring Boot

Kafka

Confluent Cloud

NGIP

Flink SQL

Schema Registry

HTTP/API integration

Kong Gateway

HTTP Sink

Payload mapping

Retries

Failure handling

Tools

GitLab CI/CD

Kong Gateway

Jobbeschreibung

Must Have:

• Develop custom integration code using Java / Spring Boot for source system connectivity where managed connectors are not applicable.
• Build and maintain Kafka producers / consumers, topic integration logic, and event-driven services on Confluent Cloud / NGIP.
• Develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
• Implement integration with Schema Registry, topic schemas, and message validation controls. Support HTTP/API-based integrations through Kong Gateway for NGIP to target application communication.
• Configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling.

Good To Have:

• Experience with stream governance, RBAC, and tiered storage
• Background in data engineering, ETL, or microservices architecture
• Familiarity with traditional messaging systems and data warehouses
• Experience in Agile methodologies and DevOps practice.

SN Responsibility of / Expectations from the Role

1. Develop custom integration code using Java / Spring Boot for source system connectivity where managed connectors are not applicable
2. Build and maintain Kafka producers / consumers, topic integration logic, and event-driven services on Confluent Cloud / NGIP.
3. Develop and maintain Flink SQL / stream processing logic for transformation, enrichment, validation, routing, and de-duplication.
4. Implement integration with Schema Registry, topic schemas, and message validation controls..
5. Support HTTP/API-based integrations through Kong Gateway for NGIP to target application communication
6. Configure and support HTTP Sink delivery patterns, payload mapping, retries, and failure handling
7. Perform unit / component testing for custom code, Flink jobs, and interface logic.
8. Support SIT, pipeline testing, defect fixing, replay, and root-cause analysis during test execution
9. Support E2E testing with downstream teams and resolve integration issues across connectors, transformations, APIs, and Kafka topics
10. Work with GitLab CI/CD pipelines for build, deployment, regression automation, and release support