DATA ENGINEERING MANAGER

Zoho; Dadri; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Vrinda International is seeking a seasoned Data Engineering Manager to lead our data infrastructure initiatives within the IT Services sector. This role oversees design, implementation, and optimization of scalable data pipelines and architectures, guiding decisions on data lake frameworks, real-time streaming, and cloud-native deployment to ensure robust analytics capabilities.

The candidate will mentor senior engineers, drive data governance, and align roadmaps with business goals while

Qualifications

• 10–15 years of data engineering experience with complex data ecosystems.
• Deep expertise in streaming and batch processing architectures (Kafka/MSK).
• Hands-on with Docker, ECS, and EKS for containerized data services.
• Strong infra automation using Terraform or AWS CDK.
• Extensive background in Data Lake architectures, Avro, Debezium CDC.
• Familiarity with real-time analytics engines like Apache Pinot desirable.

Responsibilities

• Architect and govern large-scale data lake environments for batch and real-time workloads.
• Lead streaming data solutions using Kafka/MSK and Debezium CDC integration.
• Manage containerized data services with ECS/EKS and Docker for deployment and scaling.
• Automate provisioning with Terraform and AWS CDK to optimize cloud resources.
• Define data engineering roadmaps and enforce data governance best practices.
• Mentor senior engineers and foster technical excellence within the team.

Skills

Data engineering

Apache Kafka

MSK

Docker

AWS CDK

Terraform

Data Lake

Pinot

Team leadership

Tools

ECS

EKS

Kafka/MSK

Debezium

Job description

Welcome to Vrinda International's Talent Portal

Discover your next career milestone with Vrinda International, a leading recruitment firm specializing in IT talent acquisition, international education consultancy, and language coaching opportunities. Our portal streamlines your job search with cutting-edge Applicant Tracking System (ATS) technology.

Why Choose Us?

• Seamless Candidate Experience: Upload your resume, browse live openings in Java, Spring Boot, microservices, Angular, and more – or explore roles in global education programs (Germany, UK, France).
  
• Advanced Matching: Our AI-driven tools match your skills to top IT, business development, and overseas study consultancy positions.
  
• Easy Application Process: One-click apply, real-time status updates, and integration with LinkedIn for effortless profile imports.
  
• Exclusive Opportunities: Access hidden job markets, visa guidance for international roles, and personalized career coaching.
  

Join thousands of professionals who've advanced their careers through Vrinda International. Create your profile today and let's connect your talent to global opportunities!

Job Description

Summary

We are seeking a seasoned Data Engineering Manager to lead our data infrastructure initiatives within the IT Services sector. This pivotal role involves overseeing the design, implementation, and optimization of scalable data pipelines and architectures. The successful candidate will drive strategic decisions regarding data lake frameworks, real-time streaming solutions, and cloud-native deployment strategies, ensuring robust data availability and integrity to support enterprise-wide analytics and business intelligence goals.

Responsibilities

• Architect and govern large-scale data lake environments, ensuring high performance and reliability for both batch and real-time data processing workloads.
  
• Lead the development and maintenance of streaming data solutions using Apache Kafka and MSK, integrating change data capture (CDC) mechanisms via Debezium.
  
• Manage containerized data services and orchestration platforms, specifically utilizing ECS, EKS, and Docker to ensure seamless deployment and scaling.
  
• Oversee infrastructure-as-code practices using Terraform and CDK to automate provisioning and manage cloud resources efficiently.
  
• Direct the implementation of Avro serialization standards and optimize data ingestion patterns for complex analytical requirements.
  
• Collaborate with cross-functional teams to define data engineering roadmaps, resolve technical bottlenecks, and enforce best practices in data governance.
  
• Mentor senior engineers and foster a culture of technical excellence, code quality, and continuous improvement within the data engineering team.
  

Requirements

Requirements:

• Possess 10 to 15 years of professional experience in data engineering, with a proven track record of managing complex data ecosystems.
  
• Demonstrate deep expertise in streaming and batch processing architectures, with hands-on experience in Apache Kafka and MSK.
  
• Proficiency in containerization and orchestration tools, including Docker, ECS, and EKS.
  
• Strong command of infrastructure automation using Terraform or AWS CDK.
  
• Extensive background in designing and maintaining Data Lake architectures, with specific knowledge of Avro and Debezium CDC.
  
• Familiarity with real-time analytics engines such as Apache Pinot is highly desirable.
  
• Ability to operate within the IT Services industry standards, delivering high-quality solutions that meet client and business needs.