Senior Data Engineer

Movate; Bengaluru; On-site; INR 1,200,000 - 2,500,000; Full time

Job summary

Movate is seeking a Data Engineering professional to work on a Hadoop-based data warehouse, building and maintaining large data pipelines for analytics and insights.

You will design, develop, and optimize robust data pipelines, ensure data reliability and performance, and contribute to architecture reviews and incident response.

Qualifications

• Minimum 5 years of experience in Data Engineering.
• Design, develop, and maintain robust data pipelines in Hadoop ecosystem.
• Implement data ETL processes for batch and streaming analytics.
• Optimize and troubleshoot distributed ingestion, storage, and processing systems.
• Collaborate with data analysts and platform engineers; align solutions with business needs.
• Ensure data security, integrity, and compliance throughout the infra.
• Maintain documentation and contribute to architecture reviews.
• Participate in incident response and operational excellence.
• Continuously learn Hadoop ecosystem tools and data technologies.

Responsibilities

• Design and implement scalable data pipelines for analytics workloads.
• Maintain Hadoop-based data warehouse and related infrastructure.
• Develop ETL/ELT processes for batch and streaming data.
• Optimize performance of distributed data processing jobs.
• Collaborate with data engineers, analysts, and platform teams.
• Ensure data quality, security, and regulatory compliance.
• Document architecture, pipelines, and best practices.
• Participate in incident response and reliability initiatives.

Skills

SQL

Python

PySpark

ETL/ELT Development

CDC

Medallion Architecture

Dimensional Modeling

SCD Type 1

SCD Type 2

HANA migration to Snowflake

Education

Bachelor's degree in Computer Science, IT, Engineering, or related field

Tools

Snowflake Architecture

Snowpipe

Streams & Tasks

Virtual Warehouses

dbt

Snowpark

Job description

Looking for a Data Engineering professional to work on our Hadoop-based data warehouse, contributing to scalable and reliable big data solutions for analytics and business insights. You will be building, optimizing, and maintaining large data pipelines and warehouse infrastructure in this role.

Key Responsibilities

• Minumum 5 years of experince in Data Engineering.
• Design, develop, and maintain robust data pipelines in Hadoop and related ecosystems, ensuring data reliability, scalability, and performance.
• Implement data ETL processes for batch and streaming analytics requirements.
• Optimize and troubleshoot distributed systems for ingestion, storage, and processing.
• Collaborate with data engineers, analysts, and platform engineers to align solutions with business needs.
• Ensure data security, integrity, and compliance throughout the infrastructure.
• Maintain documentation and contribute to architecture reviews.
• Participate in incident response and operational excellence initiatives for the data warehouse.
• Continuously learn mindset and apply new Hadoop ecosystem tools and data technologies.

Required Skills

• Snowflake Architecture, Snowpipe, Streams & Tasks, Virtual Warehouses.
• SQL, Python, PySpark.
• ETL/ELT Development and CDC.
• dbt.
• Medallion Architecture.
• Dimensional Modeling and SCD Type 1 & Type 2.
• SAP HANA to Snowflake migration experience.
• Strong analytical and stakeholder management skills.

Preferred Qualifications

• Bachelor's degree in Computer Science, IT, Engineering, or related field.
• Cloud certifications (AWS/Azure/GCP).

Nice to Have

• Snowpark
• LLMs, Prompt Engineering, RAG
• Experience with globally distributed teams