Data Engineer

Maveric; Pune District, Chennai District, Bengaluru; On-site; INR 2,800,000 - 4,200,000; Full time

Job summary

Maveric seeks an experienced Azure Data Engineer to design, develop, and maintain scalable data pipelines and data warehouses. The role focuses on integrating Azure services with Snowflake and AWS, building data lakes using ADLS, and optimizing data workflows with PySpark in Databricks.

The ideal candidate will have 5+ years in data engineering, strong cloud experience, and a solid foundation in governance, security, and performance tuning.

Qualifications

• Bachelor's degree in Computer Science, IT, Engineering, or related field.
• 5+ years of experience in Data Engineering.
• Minimum 3+ years of hands-on experience with Azure Data Engineering technologies.
• Strong experience with Snowflake and Databricks.
• Experience working in multi-cloud environments (Azure and AWS).
• Expertise in ETL/ELT framework development and data warehouse concepts.
• Strong understanding of data governance, security, and cloud best practices.
• Excellent analytical and problem-solving skills.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Azure and Databricks.
• Build scalable ETL/ELT processes to ingest, transform, and load data from multiple sources.
• Develop and optimize data warehouse solutions using Snowflake.
• Implement data integration solutions across Azure and AWS cloud platforms.
• Create and manage data lakes using ADLS.
• Develop data transformation logic using PySpark, Spark SQL, and Databricks notebooks.
• Optimize data processing performance, storage, and query execution.
• Automate data workflows using ADF and Databricks Jobs.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Implement data governance, security, and compliance best practices.
• Monitor and troubleshoot production data pipelines and resolve bottlenecks.
• Participate in architecture discussions and recommend best practices.

Skills

ETL/ELT development

Multi-cloud experience

Python

SQL

Spark

Data governance

Security

Problem solving

Education

Bachelor's degree in Computer Science / IT / Engineering

Tools

Azure Data Factory

Azure Databricks

ADLS Gen2

Azure Synapse Analytics

Azure SQL Database

Azure Key Vault

Azure DevOps

Azure Monitor

Snowflake

AWS Glue

S3

Redshift

Databricks

PySpark

Job description

Job Summary

We are seeking a highly skilled Azure Data Engineer with strong experience in Snowflake, AWS, Databricks, and Azure Cloud technologies. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, data warehouses, and cloud-based analytics solutions. The candidate should have expertise in building modern data platforms using Azure services while integrating data from AWS environments and Snowflake ecosystems.

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Azure and Databricks.
• Build scalable ETL/ELT processes to ingest, transform, and load data from multiple sources.
• Develop and optimize data warehouse solutions using Snowflake.
• Implement data integration solutions across Azure and AWS cloud platforms.
• Create and manage data lakes using Azure Data Lake Storage (ADLS).
• Develop data transformation logic using PySpark, Spark SQL, and Databricks notebooks.
• Optimize data processing performance, storage, and query execution.
• Automate data workflows using Azure Data Factory (ADF) and Databricks Jobs.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Implement data governance, security, and compliance best practices.
• Monitor and troubleshoot production data pipelines and resolve performance bottlenecks.
• Participate in architecture discussions and recommend best practices for cloud data solutions.

Required Skills

Azure Technologies

• Azure Data Factory (ADF)
• Azure Databricks
• Azure Data Lake Storage (ADLS Gen2)
• Azure Synapse Analytics
• Azure SQL Database
• Azure Key Vault
• Azure DevOps
• Azure Monitor

Snowflake

• Snowflake Data Warehouse
• Snowflake Architecture & Security
• Snowpipe
• Streams & Tasks
• Data Sharing
• Performance Tuning & Query Optimization

AWS Technologies

• Amazon S3
• AWS Glue
• AWS Lambda
• AWS EMR (Preferred)
• Amazon Redshift (Preferred)
• IAM and Security Management

Databricks

• PySpark
• Spark SQL
• Delta Lake
• Databricks Workflows
• Unity Catalog
• Performance Optimization

Programming & Database Skills

• Python
• SQL
• Shell Scripting
• Spark
• Data Modeling (Star Schema, Snowflake Schema)
• Relational Databases (SQL Server, Oracle, PostgreSQL)

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.
• 5+ years of experience in Data Engineering.
• Minimum 3+ years of hands-on experience with Azure Data Engineering technologies.
• Strong experience with Snowflake and Databricks.
• Experience working in multi-cloud environments (Azure and AWS).
• Expertise in ETL/ELT framework development and data warehouse concepts.
• Strong understanding of data governance, security, and cloud best practices.
• Excellent analytical and problem-solving skills.

Preferred Qualifications

• Azure Data Engineer Associate Certification (DP-203).
• SnowPro Core Certification.
• AWS Certified Data Engineer or AWS Solutions Architect Certification.
• Experience with CI/CD implementation using Azure DevOps.
• Knowledge of Airflow, Terraform, or Infrastructure as Code (IaC).

Nice to Have

• Experience with streaming technologies such as Kafka or Event Hubs.
• Experience in Banking, Financial Services, Healthcare, or Retail domains.
• Knowledge of Machine Learning data pipelines.
• Exposure to real-time analytics and big data ecosystems.