Kafka Platform Engineer

Spectraforce Technologies, Inc.; Austin (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

Spectraforce Technologies, Inc. is seeking a Kafka Platform Engineer for an on-site, 12-month assignment in Austin. The role emphasizes designing secure Kafka architectures, HA/DR, and enterprise-grade data integration, with AI-enabled development and DevOps capabilities.

You will work across data streaming, AI development, and automation, collaborating with cross-functional teams to deliver scalable distributed systems and reliable operations.

Qualifications

• 10+ years of experience in distributed systems and enterprise platform engineering.
• Extensive hands on experience with Confluent Kafka Platform (on prem and cloud).
• Strong background in Linux/Unix environments, networking, OS performance, and security.
• Proven experience with automation, DevOps, and operational tooling.
• Experience leveraging AI powered tools to accelerate development and improve team productivity and efficiency.
• Excellent written and verbal communication skills.
• Strong organizational, relationship management, and teamwork skills.

Responsibilities

• Design and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolation.
• Define and support high availability, disaster recovery, and data replication strategies.
• Ensure alignment with enterprise security, compliance, and governance standards.
• Advise on data ingestion, event streaming, batch and stream processing architectures.
• Provide guidance on data modeling and serialization standards (Avro, Protobuf, JSON Schema).
• Support enterprise integration use cases and resolve complex Kafka related integration challenges.
• Design and build AI agents, custom skills, or equivalent frameworks to automate engineering workflows at scale.
• Leverage AI-powered tools to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows.
• Lead automation first operations using Python, Ansible, and scripting.
• Build and maintain self service operational tools to reduce day to day support overhead.
• Integrate Kafka platform operations with DevOps and CI/CD tooling.
• Lead deep dive investigations into production issues and outages.
• Perform root cause analysis, document findings, and drive remediation to closure.
• Act as trusted technical advisor to Schwab engineering, operations, and leadership teams.
• Communicate complex technical concepts clearly to technical and non-technical stakeholders.

Skills

Confluent Kafka

On-Prem & Cloud

Linux/Unix

Automation & DevOps

AI-assisted development

Incident management

Communication skills

Security & Networking

GitHub Copilot

Distributed systems

Tools

Azure

Jenkins

Puppet

Chef

Ansible

GitHub Copilot

Job description

Role: Kafka Platform Engineer

Duration: 12 Months

Location: On-Site in Austin 4 days per week

Top 3 must-have skills:

• Experience with Confluent Kafka Platform
• Experience with both On-Prem and Cloud
• Must have experience leveraging AI powered tools and building AI driven agentsls as it relates to

• Design and implement secure Kafka architectures, including authentication, authorization, encryption, and network isolation.
• Define and support high availability, disaster recovery, and data replication strategies.
• Ensure alignment with enterprise security, compliance, and governance standards.

Data Streaming & Integration

• Advise on data ingestion, event streaming, batch and stream processing architectures.
• Provide guidance on data modeling and serialization standards (Avro, Protobuf, JSON Schema).
• Support enterprise integration use cases and resolve complex Kafka related integration challenges.

AI Development

• Design and build AI agents, custom skills, or equivalent frameworks to automate engineering and operational workflows at scale
• Leverage AI-powered tools (e.g., Copilot, AI agents) to accelerate development, automate testing, improve code quality, and enable intelligent SDLC workflows aligned to AI-DLC vision.
• Design agentic workflows and multi-agent orchestration patterns to solve complex distributed systems, integration, and operational challenges
• Strong hands-on experience with GitHub Copilot and other AI-assisted development tools, including advanced prompt engineering and workflow integration to accelerate engineering
• Proven ability to leverage AI-assisted development (e.g., Copilot, code generation) and build AI-driven agents/workflows to improve team productivity and engineering efficiency

Automation & DevOps Enablement

• Drive automation first operations using Python, Ansible, and scripting.
• Build and maintain self service operational tools to reduce day to day support overhead.
• Integrate Kafka platform operations with DevOps and CI/CD tooling (e.g., Azure, Jenkins, Puppet, Chef).

Incident Management & Problem Resolution

• Lead deep dive investigations into complex production issues and outages.
• Perform root cause analysis, document findings, and drive remediation to closure.
• Operate effectively in high pressure outage and incident management scenarios.

Advisory & Stakeholder Engagement

• Act as a trusted technical advisor to Schwab engineering, operations, and leadership teams.
• Communicate complex technical concepts clearly to both technical and non technical stakeholders.
• Demonstrate strong customer focus, ownership, and onshore engagement.
• Collaborate effectively with cross functional teams and vendors.

Required Experience & Skills

• 10+ years of experience in distributed systems, messaging, and enterprise platform engineering.
• Extensive hands on experience with Confluent Kafka Platform (on prem and cloud).
• Strong background in Linux/Unix environments, networking, OS performance, and security.
• Proven experience with automation, DevOps, and operational tooling.
• Experience leveraging AI powered tools to accelerate development and improve team productivity and efficiency
• Excellent written and verbal communication skills.
• Strong organizational, relationship management, and teamwork skills.

Preferred Qualifications

• Confluent Kafka and/or Cloud certifications.
• Prior experience supporting large scale, regulated enterprise environments.