Azure Data Engineer

Shell Infotech; Hyderabad, Coimbatore District, Bengaluru; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

Shell Infotech is seeking an experienced Azure Data Engineer to design, build, and optimize scalable data pipelines on the Azure platform. The ideal candidate brings strong Python, PySpark, Azure Databricks, ADF, SQL skills and solid DevOps experience.

Responsibilities include ETL/ELT development, CI/CD with GitHub Actions, and deployment management on OpenShift with HELM. Collaboration with business and tech teams is essential to deliver robust data solutions.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or related field.
• 8-12 years of relevant experience in data engineering or similar roles.
• Hands-on experience with Azure data services and DevOps practices.

Responsibilities

• Design and develop scalable ETL/ELT data pipelines using Python and PySpark.
• Build and maintain data processing solutions with Azure Databricks and Azure Data Factory.
• Develop complex SQL queries and optimize database performance.
• Implement CI/CD pipelines using GitHub Actions.
• Manage deployments and infrastructure on OpenShift with HELM.
• Configure monitoring dashboards with Grafana for data pipelines.
• Ensure data quality through testing, monitoring, and automation.
• Collaborate with business and technical teams to deliver robust data solutions.

Skills

Python

PySpark

Azure Databricks

Azure Data Factory

SQL

ETL/ELT Development

Azure SQL Server

Azure Key Vault

GitHub Actions

CI/CD Pipelines

OpenShift

HELM

Grafana

Data Pipeline Development

Performance Optimization

Education

Bachelor's or Master's Degree in Computer Science, Engineering, Data Science, or a related field

Tools

OpenShift

HELM

Grafana

Job description

Hi Everyone,

Hope you're doing well!

We have an urgent opening for an Azure Data Engineer with one of our esteemed clients.

Job Details

• Role: Azure Data Engineer
• Experience: 8-12 Years
• Location: Bangalore, Coimbatore, Hyderabad (5 Days Work From Office)
• Employment Type: Full-Time
• Education: Bachelor's or Master's Degree in Computer Science, Engineering, Data Science, or a related field

Mandatory Skills(Required skills in the resume)

• Python
• PySpark
• Azure Databricks
• Azure Data Factory (ADF)
• SQL
• ETL/ELT Development
• Azure SQL Server
• Azure Key Vault
• GitHub Actions
• CI/CD Pipelines
• OpenShift
• HELM
• Grafana
• Data Pipeline Development
• Performance Optimization

Job Description

We are looking for an experienced Azure Data Engineer to design, build, and optimize scalable data pipelines and data solutions on the Azure platform. The ideal candidate should have strong expertise in Python, PySpark, Azure Databricks, ADF, SQL, and DevOps practices.

Key Responsibilities

• Design and develop scalable ETL/ELT data pipelines using Python and PySpark.
• Build and maintain data processing solutions using Azure Databricks and Azure Data Factory.
• Develop complex SQL queries and optimize database performance.
• Implement CI/CD pipelines using GitHub Actions.
• Manage deployments and infrastructure on OpenShift using HELM.
• Configure monitoring and observability dashboards using Grafana.
• Ensure data quality through testing, monitoring, and automation.
• Collaborate with business and technical teams to deliver robust data solutions.

Preferred Skills

• Azure Data Engineer Associate Certification
• Kafka
• Azure Event Hubs
• Data Governance & Security
• Real-time Data Processing

Interested Candidates

Contact Person: Raghavendra (HR)

Shell Infotech