Data Engineer

Vrinda International; Bengaluru, Mumbai; On-site; INR 4,500,000 - 7,000,000; Full time

Job summary

Vrinda International is seeking an experienced Data Engineer to architect and maintain scalable data infrastructure within the IT Services sector. You will design robust data warehousing and data modeling solutions, leveraging Google Cloud technologies to transform raw data into actionable insights.

You will manage cloud-native pipelines, oversee migration strategies, and optimize processing workflows, enabling data-driven decision making and operational efficiency across global client projects.

Qualifications

• 15–20 years of professional data engineering experience.
• Strong Data Warehousing and data modeling expertise.
• Proficient with Git/GitHub and CI/CD practices.
• Hands-on with Google Dataflow and Apache Beam.
• Proficient with Google BigQuery and Dataproc.
• Experience with Cloud Composer (Airflow) and Pub/Sub.
• Experience migrating data to cloud-native platforms with security and governance.
• Ability to design scalable data architectures for analytics.

Responsibilities

• Design and implement scalable data warehousing architectures.
• Develop and maintain version control and CI/CD pipelines.
• Build data pipelines with Dataflow and Beam for real-time and batch processing.
• Lead cloud migration initiatives ensuring data integrity and security.
• Manage data storage and processing using BigQuery and Dataproc.
• Orchestrate workflows with Cloud Composer and Airflow.
• Implement event-driven data ingestion using Pub/Sub and Cloud Storage.

Skills

Data Warehousing

Data Modeling

Google Cloud Dataflow

Apache Beam

BigQuery

Dataproc

Git/GitHub

CI/CD

Tools

Git

GitHub

Airflow

CI/CD pipelines

Job description

Welcome to Vrinda International's Talent Portal

Discover your next career milestone with Vrinda International, a leading recruitment firm specializing in IT talent acquisition, international education consultancy, and language coaching opportunities. Our portal streamlines your job search with cutting-edge Applicant Tracking System (ATS) technology.

Why Choose Us?

• Seamless Candidate Experience: Upload your resume, browse live openings in Java, Spring Boot, microservices, Angular, and more – or explore roles in global education programs (Germany, UK, France).
  
• Advanced Matching: Our AI-driven tools match your skills to top IT, business development, and overseas study consultancy positions.
  
• Easy Application Process: One-click apply, real-time status updates, and integration with LinkedIn for effortless profile imports.
  
• Exclusive Opportunities: Access hidden job markets, visa guidance for international roles, and personalized career coaching.
  

Job Description

Summary

We are seeking an experienced Data Engineer to architect, build, and maintain scalable data infrastructure within the IT Services sector. In this pivotal role, you will drive the design of robust data warehousing solutions and ensure seamless data modeling to support critical business analytics. Your expertise will be essential in managing cloud-native data pipelines, overseeing high-quality data migration strategies, and optimizing large-scale processing workflows. This position offers the opportunity to leverage cutting-edge Google Cloud technologies to transform raw data into actionable insights, directly contributing to the organization's data-driven decision-making capabilities and operational efficiency.

Responsibilities

• Design and implement comprehensive data warehousing architectures and advanced data modeling strategies to meet evolving business needs.
  
• Develop and maintain efficient version control workflows using Git and GitHub, integrating continuous integration and continuous deployment (CI/CD) pipelines for automated testing and deployment.
  
• Construct and optimize complex data processing pipelines utilizing Google Dataflow and Apache Beam for real‑time and batch data transformation.
  
• Establish rigorous data quality frameworks and lead strategic cloud migration initiatives to ensure data integrity and security during transitions.
  
• Manage and scale data storage and processing environments using BigQuery and Dataproc, ensuring high performance and cost‑effectiveness.
  
• Orchestrate scheduled workflows and dependency management through Cloud Composer (Apache Airflow) to guarantee reliable data delivery.
  
• Implement event‑driven data ingestion and storage solutions leveraging Pub/Sub and Cloud Storage to handle high‑volume data streams.
  

Requirements

Requirements:

• Experience: A minimum of 15 to 20 years of professional experience in data engineering and related fields.
  
• Core Competencies: Proven expertise in Data Warehousing and sophisticated Data Modeling techniques.
  
• Version Control & DevOps: Strong proficiency with Git/GitHub and the implementation of CI/CD practices.
  
• Big Data Processing: Hands‑on experience with Google Dataflow and Apache Beam for scalable data processing.
  
• Data Governance: Demonstrated ability in ensuring Data Quality standards and executing successful Cloud Migration projects.
  
• Cloud Technologies: In‑depth knowledge of Google BigQuery and Dataproc for analytics and processing.
  
• Workflow Orchestration: Skilled in utilizing Cloud Composer (Apache Airflow) for workflow management.
  
• Messaging & Storage: Familiarity with Pub/Sub and Cloud Storage for data ingestion and archival.