Kafka Specialist

Lite Thinking company; Ciudad de México; Presencial; MXN 900.000 - 1.200.000; Jornada completa

Descripción de la vacante

Lite Thinking seeks an experienced Kafka Platform Engineer to manage, configure, and optimize our Apache Kafka setup. This role will lead the ZooKeeper-to-KRaft migration, implement robust security with SASL/OAuthBearer, and enhance high availability of the Schema Registry across integrations with Teradata, MQ, and HDFS.

The ideal candidate will have a strong background in distributed systems, real-time processing, and collaboration with security, infrastructure, architecture, and development

Formación

• Experience with Apache Kafka and Confluent Platform.
• Configuring, maintaining, and monitoring Kafka clusters.
• Experience migrating ZooKeeper to KRaft.

Responsabilidades

• Administer and configure the Apache Kafka platform.
• Monitor availability, capacity, security, and performance of Kafka clusters.
• Execute migration from ZooKeeper to KRaft ensuring continuity.

Conocimientos

Apache Kafka

Confluent Platform

KSQL/ksqlDB

Kafka Streams (KStreams)

Kafka REST Proxy

ZooKeeper to KRaft migration

SASL/OAuthBearer security

Schema Registry HA

Teradata TPT

MQ Connector

Teradata HDFS Connector

Educación

Bachelor's degree in Systems/Software Engineering

Computer Science/ Telecommunications background

Herramientas

Teradata TPT

MQ Connector

Teradata Connector

HDFS 3 Connector

Descripción del empleo

Purpose of the Role

Manage, configure, and optimize the Apache Kafka platform, ensuring its availability, security, performance, and proper integration with the organization’s various systems. The role will be responsible for executing the migration from ZooKeeper to KRaft, implementing security mechanisms, strengthening the high availability of the Schema Registry, and configuring connectors for Kafka’s integration with platforms such as Teradata, MQ, and HDFS.

Ideal Candidate Profile

Professional or technologist with a degree in Systems Engineering, Software Engineering, Computer Science, Telecommunications, or related fields. An analytical, organized individual focused on solving technical problems. Ability to manage distributed platforms and high-availability environments. Focus on the stability, security, and performance of solutions. Ability to work effectively with development, architecture, infrastructure, and security teams.

Required Experience and Knowledge

• Experience working with Apache Kafka and the Confluent Kafka Platform.
• Experience in administering, configuring, maintaining, and monitoring Kafka clusters.
• Knowledge of KSQL (ksqlDB).
• Experience developing or administering solutions with Kafka Streams (KStreams).
• Knowledge of implementing and configuring the Kafka REST Proxy.
• Experience or solid knowledge of migrating from ZooKeeper to KRaft.
• Experience implementing security mechanisms using SASL and OAuthBearer.
• Knowledge of configuring, enabling, and optimizing high availability for Schema Registry.
• Experience working with Teradata TPT.
• Knowledge of configuring and integrating the MQ Connector.
• Experience with Teradata Connector.
• Knowledge of implementing and administering the HDFS 3 Connector.
• Experience diagnosing and troubleshooting issues related to performance, availability, and connectivity in Kafka.
• Knowledge of event-driven architectures, real-time data processing, and distributed systems.

Key Responsibilities

• Administer and configure the Apache Kafka platform on the Confluent Kafka Platform.
• Monitor the availability, capacity, security, and performance of Kafka clusters.
• Execute the migration from ZooKeeper to KRaft, ensuring the continuity and stability of the platform.
• Design and execute technical tests before, during, and after the migration process.
• Implement and configure authentication and security mechanisms using SASL and OAuthBearer.
• Enable and optimize high availability for the Schema Registry.
• Configure, maintain, and troubleshoot issues related to the Kafka REST Proxy.
• Implement and manage processing flows using KSQL (ksqlDB) and Kafka Streams.
• Configure and maintain integrations using MQ Connector, Teradata Connector, and HDFS 3 Connector.
• Manage data transfer and integration processes using Teradata TPT.
• Analyze and troubleshoot issues related to latency, availability, performance, and resource consumption.
• Apply best practices for configuring topics, partitions, replicas, producers, and consumers.
• Participate in performance, recovery, and high-availability testing.
• Document configurations, migrations, integrations, tests, and operational procedures.
• Work closely with the architecture, infrastructure, development, data, and security teams.

Desired Skills

• Analytical thinking and diagnostic skills.
• Ability to solve complex technical problems.
• Focus on service stability and continuity.
• Attention to detail.
• Ability to plan and execute migrations.
• Ability to collaborate with multidisciplinary teams.
• Clear communication and technical documentation.
• Organization and commitment to meeting deadlines.
• Adaptability and a commitment to continuous learning.

Contract Terms

Location: Mexico City (CDMX).Compensation: To be determined based on experience and profile.