Data Engineer - Real-Time Streaming

GSSTech Group; Chennai District; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

GSSTech Group is seeking a Data Engineer with strong expertise in real-time streaming and event-driven architectures. The role involves building scalable data pipelines, optimizing streaming systems, and ensuring high performance across distributed environments.

You will design and implement streaming pipelines using Apache Flink, Kafka, and Java; deploy applications with Docker and Kubernetes; and collaborate with DevOps for CI/CD workflows on GitHub and Jenkins.

Qualifications

• Strong hands-on experience in Real-Time Streaming (Flink, Kafka, Java).
• Deep understanding of event-driven architecture.
• Experience with JVM performance tuning.
• Proficiency in Docker and Kubernetes.
• Solid experience in Linux OS and shell scripting.
• Knowledge of design patterns and scalable system design.
• Experience with CI/CD tools like GitHub and Jenkins.
• Hands-on troubleshooting experience in distributed systems.
• Nice to have Redis or other caching systems.
• Exposure to Cloudera Data Platform engineering.
• Prior banking or financial domain experience is a plus.

Responsibilities

• Design and implement real-time data streaming pipelines using Apache Flink, Kafka, and Java.
• Build and maintain event-driven architectures for large-scale distributed systems.
• Perform JVM tuning and performance optimization for streaming applications.
• Develop and deploy applications using containerization tools (Docker, Kubernetes).
• Work with Cloudera platform for data engineering and pipeline orchestration.
• Implement robust design patterns and ensure high-quality coding standards.
• Troubleshoot and resolve issues in distributed systems ecosystem.
• Collaborate with DevOps teams to maintain CI/CD pipelines (GitHub, Jenkins).
• Work on Linux-based systems, including configuration and shell scripting.
• Optimize data processing with caching mechanisms (e.g., Redis).

Skills

Real-time streaming

Event-driven arch

JVM tuning

Docker & Kubernetes

Linux shell

System design

CI/CD

Distributed systems

Tools

Apache Flink

Kafka

Java

Docker

Kubernetes

GitHub

Jenkins

Job description

We are looking for a highly skilled?Data Engineer?with strong expertise in?real-time streaming and event-driven architectures. The ideal candidate will be responsible for building scalable data pipelines, optimizing streaming systems, and ensuring high performance across distributed environments.

Key Responsibilities

• Design and implementreal-time data streaming pipelinesusing technologies likeApache Flink, Kafka, and Java
• Build and maintainevent-driven architecturesfor large-scale distributed systems
• PerformJVM tuning and performance optimizationfor streaming applications
• Develop and deploy applications usingcontainerization tools (Docker, Kubernetes)
• Work withCloudera platformfor data engineering and pipeline orchestration
• Implement robustdesign patternsand ensure high-quality coding standards
• Troubleshoot and resolve issues indistributed systems ecosystem
• Collaborate with DevOps teams to maintainCI/CD pipelines (GitHub, Jenkins)
• Work onLinux-based systems, including configuration and shell scripting
• Optimize data processing with caching mechanisms (e.g., Redis ? nice to have)

Required Skills & Experience

• Strong hands-on experience inReal-Time Streaming (Flink / Kafka / Java)
• Deep understanding ofevent-driven architecture
• Experience withJVM performance tuning
• Proficiency inDocker and Kubernetes
• Solid experience inLinux OS and shell scripting
• Knowledge ofdesign patterns and scalable system design
• Experience withCI/CD toolslike GitHub and Jenkins
• Hands-on troubleshooting experience indistributed systems

Nice to Have

• Experience withRedis or other caching systems
• Exposure toCloudera Data Platform engineering
• Prior experience in banking or financial domain is a plus