Remote Streaming Data Engineer for Real-time AI Training

ixolabs.ai; France; À distance; EUR 35 000 - 52 000; Temps partiel

Avantages offerts par ce poste

Remote work

Flexible hours

Weekly payments via Stripe

Résumé du poste

ixolabs.ai is seeking a Streaming Data Engineer for AI training to design and review real-time data pipelines. You will craft streaming patterns, validate windowing and event-time semantics, and help optimize schema management across Confluent and Avro/Protobuf ecosystems.

You will collaborate on cross-framework implementations (Kafka, Flink, Kafka Streams, ksqlDB) and contribute to scalable, fault-tolerant streaming solutions with flexible hours.

Qualifications

• 5+ years building streaming systems in production.
• Deep familiarity with Kafka and at least one stream-processing framework (Flink, Kafka Streams, or Spark Structured Streaming).
• Strong grasp of distributed-systems concepts (CAP, exactly-once, watermarks).
• Experience with schema management and event-driven architecture patterns.
• Comfort with Java/Scala on the JVM, plus operational tools (Cruise Control, Strimzi).
• Familiarity with Pulsar, Redpanda, or AWS MSK is a plus.

Responsabilités

• Generate and evaluate streaming instruction-response pairs across Kafka, Flink, Kafka Streams, and ksqlDB.
• Review AI-generated code for correct windowing, watermarking, and event-time semantics.
• Provide feedback on consumer-group design, partition strategy, and rebalance handling.
• Validate AI handling of schema registries (Confluent, Apicurio) and Avro/Protobuf evolution.
• Evaluate exactly-once patterns, idempotent producers, and transactional consumers.
• Identify subtle issues in state-store sizing, checkpointing, and reprocessing.

Connaissances

Kafka

Flink

Kafka Streams

Java/Scala

Event-time processing

Outils

Confluent Schema Registry

Apicurio

Avro

Protobuf

Cruise Control

Strimzi

Description du poste

ixolabs.ai is seeking a Streaming Data Engineer for AI training to design and review real-time data pipelines. You will craft streaming patterns, validate windowing and event-time semantics, and help optimize schema management across Confluent and Avro/Protobuf ecosystems.

You will collaborate on cross-framework implementations (Kafka, Flink, Kafka Streams, ksqlDB) and contribute to scalable, fault-tolerant streaming solutions with flexible hours.