Confluent Kafka Platform Engineer - AI Agents

Spectraforce; Austin (TX); On-site; USD 96,000 - 103,000; Full time

Job summary

Spectraforce is seeking a senior specialist for on-site engagement in Austin, TX for a 12-month duration. The role emphasizes distributed systems, Confluent Kafka, and AI-driven development practices to accelerate engineering workflows.

Expect collaboration with cross-functional teams and adherence to security and governance standards. The position requires extensive Kafka expertise, Linux proficiency, and a strong DevOps/automation background with experience in AI-powered tooling.

Qualifications

• 10+ years of experience in distributed systems, messaging, and enterprise platform engineering.
• Extensive hands-on experience with Confluent Kafka Platform (on prem and cloud).
• Strong Linux/Unix, networking, OS performance and security background.
• Experience with automation, DevOps, and operational tooling.
• Experience leveraging AI-powered tools to accelerate development and improve productivity.
• Excellent written and verbal communication; strong organizational and teamwork skills.
• Confluent Kafka and/or Cloud certifications preferred.

Responsibilities

• Design and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolation.
• Define and support HA, DR, and data replication strategies.
• Ensure alignment with security, compliance, and governance standards.
• Data Streaming & Integration: advise on ingestion and batch/stream processing.
• Support data modeling and serialization standards (Avro, Protobuf, JSON Schema).
• Lead incident management and root cause analysis for production issues.
• Act as trusted technical advisor to engineering, operations, and leadership.

Skills

Distributed systems

Technical advisory

Communication

Teamwork

Tools

Confluent Kafka Platform

Linux/Unix

Azure

Jenkins

Puppet

Chef

Job description

Duration: 12 Months

Location: On-Site in Austin, TX 4 days per week

Top 3 must-have skills:

Experience with Confluent Kafka Platform

Experience with both On-Prem and Cloud

Must have experience leveraging AI powered tools and building AI driven agentsls as it relates to

• Design and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolation.
• Define and support high availability, disaster recovery, and data replication strategies.
• Ensure alignment with enterprise security, compliance, and governance standards.
• Data Streaming & IntegrationAdvise on data ingestion, event streaming, batch and stream processing architectures.
• Provide guidance on data modeling and serialization standards (Avro, Protobuf, JSON Schema).
• Support enterprise integration use cases and resolve complex Kafka related integration challenges.
• AI DevelopmentDesign and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale
• Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
• Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges
• Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering
• Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency
• Automation & DevOps EnablementDrive automation first operations using Python, Ansible, and scripting.
• Build and maintain self service operational tools to reduce day to day support overhead.
• Integrate Kafka platform operations with DevOps and CI/CD tooling (e.g., Azure, Jenkins, Puppet, Chef).
• Incident Management & Problem ResolutionLead deep dive investigations into complex production issues and outages.
• Perform root cause analysis, document findings, and drive remediation to closure.
• Operate effectively in high pressure outage and incident management scenarios.
• Advisory & Stakeholder EngagementAct as a trusted technical advisor to Schwab engineering, operations, and leadership teams.
• Communicate complex technical concepts clearly to both technical and non technical stakeholders.
• Demonstrate strong customer focus, ownership, and onshore engagement.
• Collaborate effectively with cross functional teams and vendors.
• Required Experience & Skills10+ years of experience in distributed systems, messaging, and enterprise platform engineering.
• Extensive hands on experience with Confluent Kafka Platform (on prem and cloud).
• Strong background in Linux/Unix environments, networking, OS performance, and security.
• Proven experience with automation, DevOps, and operational tooling.
• Experience leveraging AI powered tools to accelerate development and improve team productivity and efficiency
• Excellent written and verbal communication skills.
• Strong organizational, relationship management, and teamwork skills.
• Preferred QualificationsConfluent Kafka and/or Cloud certifications.
• Prior experience supporting large scale, regulated enterprise environments.
• For information on benefits, equal opportunity employment, and location-specific applicant notices, click here

At SPECTRAFORCE, we are committed to maintaining a workplace that ensures fair compensation and wage transparency in adherence with all applicable state and local laws.This position's pay range is $70.00/hr - $75.00/hr.