Analyst/Consultant - Data Engineer - Azure (INDM)

Blue Altair; Pune District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

A leading consulting firm in Pune is looking for an Analyst/Consultant - Data Engineer with 3-4 years of experience. Responsibilities include designing data pipelines, managing Azure Data Factory workflows, and ensuring data security. The ideal candidate should have proficiency in Python and PySpark, along with strong communication skills. This position offers competitive compensation and a range of employee benefits.

Qualifications

• 3-4 years of experience in data engineering.
• Proficiency in Python for scripting and automation.
• Experience with large-scale data processing using PySpark.

Responsibilities

• Design and implement data pipelines for BI delivery.
• Create Azure Data Factory pipelines for data extraction.
• Manage databases, both SQL and NoSQL.

Skills

Python

PySpark

Azure Data Factory (ADF)

Azure Databricks

Communication

Teamwork

Job description

Overview

Greetings from Blue Altair!

About Blue Altair:

Blue Altair is an innovative, industry-recognized consulting firm that leverages transformative technologies to enable AI and drive digital success for its clients. Founded in 2015, our services span Assessment and Strategy, Technology Implementation, and Managed Services across API Management and Integration, Data Management, Digital Application Development, and Artificial Intelligence. We are proud to exceed industry standards for project success, thanks to our expert focus on program and project management, business analysis, and quality assurance.

Working at Blue Altair means being part of a vibrant, dynamic team that drives digital change and empowers clients to embrace AI transformations. No matter the role you fill, you\'ll help shape the digital future and make a real difference.

So, if you\'re a star ready to shine even brighter, we can\'t wait to meet you. Join us at Blue Altair—where we\'re not just transforming businesses but shaping the AI universe one star at a time!

Title: Analyst/Consultant - Data Engineer

Experience: 3-4 years

Location: Pune

Responsibilities

• Data Pipeline Development: Design and implement data pipelines to efficiently deliver data to the Business Intelligence (BI) team.
• Create and manage Azure Data Factory (ADF) pipelines to extract data from various sources and load it into Azure Blob Storage (Bronze layer).
• Develop pipelines using Azure Databricks to process and transform data from the bronze layer into the silver layer, creating intermediate tables.
• Build and maintain pipelines to transform silver layer data into Gold layer fact and dimension tables, ensuring the data is ready for analysis by the BI team.
• Monitor, troubleshoot, and optimize all data pipelines to ensure they run smoothly and efficiently.

• Data Integration: Integrate data from diverse sources such as databases, APIs, and third-party systems.
• Develop solutions for data extraction, transformation, and loading to data warehouses or data lakes.

• Database Management: Manage and optimize databases, both relational (e.g., SQL) and NoSQL (e.g., MongoDB, Cassandra).
• Implement and maintain data models and database schemas.

• Data Governance and Security: Implement and enforce data governance policies and best practices.
• Ensure data security and compliance with relevant regulations.

Qualifications

• Technical Skills:
• Python: Proficiency in Python for scripting and automation tasks.
• PySpark: Experience with PySpark for large-scale data processing and transformations.
• Azure Data Factory (ADF): Expertise in designing and managing data workflows using Azure Data Factory.
• Azure Databricks: Hands-on experience with Azure Databricks for data engineering and pipeline development.

• Soft Skills:
• Excellent communication and teamwork abilities.
• Ability to manage multiple tasks and projects simultaneously.
• Proactive and self-motivated with a continuous learning mindset.

In addition to a competitive compensation package, we offer abundant opportunities for you to achieve, excel, and surpass even your own expectations. Aligned with Blue Altair\'s challenger ethos, we present the Star Elements program, inspired by the composition of stars themselves, comprising diverse elements that give unique qualities and illuminate for thousands of years. Likewise, our initiative aims to facilitate brilliance in our employees. The Star Elements program encompasses a range of benefits, supporting our team across four pivotal facets of their lives and careers. Through Star Elements, our employees can fully realize their potential, both on personal and professional fronts.