Kafka Platform Engineer

Procom; Austin (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Schwab is seeking a Kafka Platform Engineer for an onsite assignment in Austin, TX. You will bring 10+ years in distributed systems and deep expertise with the Confluent Kafka Platform across on‑prem and cloud environments.

In this role you will design secure Kafka architectures, drive HA/DR, assist with data replication, and lead AI-enabled development and automation initiatives to improve engineering efficiency at scale.

Qualifications

• 10+ years of experience in distributed systems, messaging, and enterprise platform engineering.
• Extensive hands-on experience with the Confluent Kafka Platform (on-prem and cloud).
• Strong background in Linux/Unix, networking, OS performance and security.
• Experience with automation, DevOps and operational tooling.
• Experience leveraging AI-powered tools to accelerate development and productivity.

Responsibilities

• Design and implement secure Kafka architectures with authentication, authorization, encryption and network isolation.
• Define and support high availability, disaster recovery and data replication strategies.
• Advise on data ingestion, event streaming and batch/stream processing architectures.
• Design and build AI agents, or equivalents, to automate engineering workflows at scale.
• Drive automation-first operations and integrate Kafka with CI/CD tooling.
• Lead deep-dive investigations into production issues and incidents.

Skills

Distributed systems

Kafka Platform

Automation

DevOps

AI tools

Communication

Leadership

Tools

Confluent Kafka Platform

GitHub Copilot

Job description

Kafka Platform EngineerAssignment DetailsLocation: Onsite, Austin, TX — 4 days/weekDuration: 12-month contractTop 3 Must-Have SkillsExperience with the Confluent Kafka PlatformExperience with both on-prem and cloud environmentsExperience leveraging AI-powered tools and building AI-driven agentsKey ResponsibilitiesSecurity & ComplianceDesign and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolationDefine and support high availability, disaster recovery, and data replication strategiesEnsure alignment with enterprise security, compliance, and governance standardsData Streaming & IntegrationAdvise on data ingestion, event streaming, and batch/stream processing architecturesProvide guidance on data modeling and serialization standards (Avro, Protobuf, JSON Schema)Support enterprise integration use cases and resolve complex Kafka-related integration challengesAI DevelopmentDesign and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scaleLeverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflowsDesign agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challengesStrong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integrationProven ability to leverage AI-assisted development and build AI-driven agents/workflows to improve team productivity and engineering efficiencyAutomation & DevOps EnablementDrive automation-first operations using Python, Ansible, and scriptingBuild and maintain self-service operational tools to reduce day-to-day support overheadIntegrate Kafka platform operations with DevOps and CI/CD tooling (e.g., Azure, Jenkins, Puppet, Chef)Incident Management & Problem ResolutionLead deep-dive investigations into complex production issues and outagesPerform root cause analysis, document findings, and drive remediation to closureOperate effectively in high-pressure outage and incident management scenariosAdvisory & Stakeholder EngagementAct as a trusted technical advisor to Schwab engineering, operations, and leadership teamsCommunicate complex technical concepts clearly to both technical and non-technical stakeholdersDemonstrate strong customer focus, ownership, and onshore engagementCollaborate effectively with cross-functional teams and vendorsRequired Experience & Skills10+ years of experience in distributed systems, messaging, and enterprise platform engineeringExtensive hands-on experience with the Confluent Kafka Platform (on-prem and cloud)Strong background in Linux/Unix environments, networking, OS performance, and securityProven experience with automation, DevOps, and operational toolingExperience leveraging AI-powered tools to accelerate development and improve team productivity and efficiencyExcellent written and verbal communication skillsStrong organizational, relationship management, and teamwork skillsPreferred QualificationsConfluent Kafka and/or cloud certificationsPrior experience supporting large-scale, regulated enterprise environments