kafka Architect

Tata Consultancy Services; City of Melbourne; On-site; AUD 100,000 - 150,000; Full time

Benefits offered by this job

Performance-based incentives

Comprehensive employee benefits

Recognition programs

Job summary

Tata Consultancy Services is seeking a skilled individual to design and operate Kafka within large-scale distributed environments. Responsibilities include leading event-driven architecture implementations and ensuring security standards are enforced. The ideal candidate will possess strong experience in real-time streaming applications and security protocols. We offer an attractive compensation package that aligns with the Australian market, along with performance incentives and comprehensive employee benefits.

Qualifications

• Intermediate to advanced knowledge of Kafka and eventing.
• Experience operating real-time streaming applications.
• Familiarity with security controls: ACLs, RBAC, and TLS.

Responsibilities

• Design, deploy, and operate Kafka in large-scale environments.
• Lead use case design for event-driven architecture.
• Support real-time data processing and streaming workloads.

Skills

Kafka & Eventing

Confluent Platform / Confluent Cloud

Stream Processing

Security & Access Controls

CI/CD & Containerisation

Tools

GKE / AKS / EKS

Terraform

New Relic

Docker

Job description

Job Description

Roles and Responsibilities

• Design, deploy, and operate Apache Kafka / Confluent Platform / Confluent Cloud in large-scale, distributed environments.
• Define and implement Kafka broker architecture, Schema Registry, Kafka Connect, and consumer group strategies.
• Establish eventing standards, schema governance, topic naming conventions, retention policies, and platform guardrails.
• Drive enterprise event-driven architecture (EDA) adoption beyond Kafka infrastructure (event design, patterns, and contracts).
• Event-Driven Design & Stream Processing
• Lead event-driven use case design and implementation, collaborating with domain and application teams.
• Design and build stream processing applications using Flink, Kafka Streams (KStreams) or equivalent frameworks (Spark Streaming, Google Dataflow, etc.).
• Support real-time data processing, enrichment, aggregation, and stateful streaming use cases.
• Deploy and manage Kafka and streaming workloads on Kubernetes (GKE preferred; AKS/EKS acceptable).
• Use Helm, Kustomize, and Kubernetes Operators for repeatable and scalable deployments.
• Integrate Kafka platforms with cloud-native services such as compute, storage, and messaging systems.
• Security, Access Control & Compliance
• Design and enforce platform and data security controls, including ACLs / RBAC, TLS/mTLS, certificate management, and secrets management.
• Implement cloud IAM, Secret Manager, and Cloud KMS (or equivalent AWS/Azure services).
• Embed code and infrastructure security using tools such as Snyk, SonarQube, Nexus, Trivy.
• Observability, Reliability & Operations
• Build and operate monitoring and observability solutions (New Relic, Sumo Logic, or similar tools such as Splunk, Datadog, AppDynamics).
• Support incident triage, root‑cause analysis, capacity planning, and performance tuning.
• Define SLOs, SLIs, and operational runbooks for Kafka and streaming platforms.
• DevOps, Automation & CI/CD
• Develop Infrastructure as Code (IaC) using Terraform.
• Implement and maintain CI/CD pipelines using GitLab CI or equivalent tooling (GitHub Actions, Jenkins, ArgoCD, Harness).
• Automate platform operations and validations using Shell/Bash and Python.
• Consulting & Enablement
• Act as a Kafka subject‑matter expert, mentoring teams and reviewing solution designs.
• Provide architectural guidance during solution design, reviews, and production readiness assessments.
• Collaborate with enterprise architects, security teams, and SRE/DevOps functions.

Desired Competencies (Technical/Behavioral)

Must‑Have

• Kafka & Eventing (Intermediate – Advanced)
• Schema Registry, Kafka Connect, Consumer Groups
• Confluent Platform / Confluent Cloud
• Event‑driven architecture standards and governance
• Stream Processing (Intermediate)
• Experience operating real‑time streaming applications
• GKE / AKS / EKS
• Security & Access Controls (Must Have)
• ACLs, RBAC
• OAuth 2.0, TLS/mTLS
• Certificate and secrets management

Observability & Monitoring (Must Have)

• New Relic, Sumo Logic, or equivalent tools
• Messaging Systems (Intermediate)
• GCP Pub/Sub or AWS SQS / Azure messaging / JMS
• Infrastructure as Code
• Terraform
• CI/CD & Containerisation
• GitLab CI or equivalent
• Docker

Good‑to‑Have

• Prior experience in enterprise integration, supply chain, retail, or logistics domains
• Exposure to multi‑region, high‑throughput Kafka deployments
• Confluent, Kubernetes, or Cloud certifications

Attractive Compensation & Rewards

We offer a highly competitive salary package aligned with the Australian market, complemented by performance‑based incentives, recognition programs, and comprehensive employee benefits. Your achievements are celebrated, and your contributions truly matter.

Equal Opportunity Employer

TCS is an equal opportunity employer. All qualified applicants will be considered without regard to race, colour, religion, gender, gender identity, sexual orientation, national origin, disability, veteran status, or any other characteristic protected by applicable law.

Note: Compensation details are accurate as of this posting and may be adjusted in line with Australian laws and market conditions. Applicants may be asked to attend interviews in person or via video conference and present valid identification during the process.