Messaging Engineer

CONSULGURU PTE. LTD.; Singapore; On-site; SGD 110,000 - 150,000; Full time

Job summary

CONSULGURU PTE. LTD.

in Singapore is seeking an experienced Messaging/Streaming Engineer to design, implement, and maintain the messaging layer that underpins real-time data exchange between factory automation systems, the MES, and enterprise IT platforms. You will deploy distributed brokers such as Apache Kafka clusters to enable streaming from shop-floor equipment to cloud applications, define topics and data schemas, and optimize for low latency and reliability.

Qualifications

• Experience deploying Kafka or similar in production.
• Designing real-time pipelines with pub/sub patterns.
• Proficiency in Java/Scala or Python.
• Understanding broker clustering and offset management.

Responsibilities

• Design, implement, and maintain the messaging infrastructure for real-time data exchange between factory automation, MES, and enterprise IT platforms.
• Deploy and manage distributed message brokers to enable streaming from shop-floor equipment to cloud applications.
• Define topics/channels and data schemas to ensure consistent event structures.
• Collaborate with developers to integrate applications with the messaging layer and tune performance.

Skills

Kafka & streaming tech

Java/Scala or Python

MQTT/RabbitMQ/IBM MQ

Pub/Sub real-time pipelines

Broker configuration / topics design

Tools

Kafka Connect

Job description

Core Responsibilities:

Design, implement, and maintain the messaging infrastructure that underpins real-time data exchange between factory automation systems, the MES, and enterprise IT platforms. Deploy and manage distributed message brokers (such as Apache Kafka clusters or MQTT-based systems) to enable streaming communication from shop-floor equipment and sensors to cloud or data center applications.

Define and govern message topics/channels and data schemas to ensure consistent structure for all events (e.g. equipment status updates, test results, alarm signals) flowing through the system.

Optimize the messaging setup for low latency and high reliability, so that critical events propagate instantly and without loss or duplication. Implement monitoring and alerting for the messaging pipeline – tracking metrics like throughput, lag, and error rates – and quickly troubleshoot any issues in message flow health.

Collaborate with other developers to integrate their applications with the messaging layer, and assist in tuning producer/consumer performance to meet factory throughput demands.

Must Have Skills:

Strong expertise in message-oriented middleware and streaming platforms. Hands-on experience deploying or managing Apache Kafka (preferred) or similar technologies (e.g. RabbitMQ, IBM MQ, Apache Pulsar) in a production environment – including topic design, partitioning, replication, and broker configuration. Proficiency in designing real-time data pipelines with pub/sub patterns, ensuring high throughput and fault tolerance (e.g. using Kafka Connect or MQTT clients to interface with devices and applications).

Solid programming skills (Java/Scala or Python) for developing producers and consumers, and optimizing them for performance.

Familiarity with messaging system internals (broker clustering, retention policies, consumer groups, offset management) to fine-tune the system for optimal delivery guarantees.

Good to Have Skills:

Experience with messaging in industrial/IoT contexts, such as using MQTT or AMQP protocols to collect data from PLCs or IoT sensors. Knowledge of data serialization formats and schema management, for example, using Avro or Protocol Buffers and a Schema Registry to enforce consistent message structure across services.

Proficiency in monitoring tools and practices for streaming systems (e.g. Kafka monitoring via Prometheus/Grafana, or cloud services monitoring) and experience setting up alerts/dashboard for proactive incident response.

Familiarity with other integration patterns (like enterprise service bus architectures or event-driven microservices) and understanding of how messaging integrates with the overall factory IT landscape (MES, databases, analytics) would be beneficial.