Azure Data Engineer

InfoVision Inc.; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

InfoVision Inc. is seeking an Azure Data Engineer to assess diverse data sources (SAP and non-SAP) and rapidly turn them into business-ready data sets on our Azure platform.

You will design and implement robust data pipelines, work with Databricks and Azure services, and collaborate across teams to deploy scalable analytics solutions.

Qualifications

• Assess complex data sources (SAP and non-SAP) and turn into ready-to-use datasets.
• Design, document, develop, implement, and test rapid prototypes showcasing data value.
• Participate in early design and feature definition activities.
• Implement robust data pipelines using Microsoft toolset and Databricks.
• Create reusable, scalable data pipelines and collaborate across teams.

Responsibilities

• Assess data sources for quality, completeness and business value.
• Design and prototype data solutions and present value to stakeholders.
• Contribute to early-stage design and feature definition discussions.
• Implement robust, scalable data pipelines using Azure and Databricks.
• Collaborate with engineers to integrate prototypes into core products.

Skills

Relational databases

Non-relational databases

Big data pipelines

Unstructured data analysis

Python

PySpark

Education

Bachelor's degree in Computer Science or allied field

Azure platform certifications (advantage)

Tools

Azure Databricks

Azure Synapse

SQL

EventHub

Python

Job description

As an Azure Data Engineer in our team, you will be responsible for assessing complex new data sources with a variety of SAP and non-SAP source system and quickly turning these into business insights ready data sets. You also will develop and support the implementation and integration of these new data sources into our Azure Data platform.

Responsibilities:

• You are required to do detailed analysis of structured, semi-structured and unstructured data sources for quality, completeness, and business value.
• You are required to design, document, develop, implement, and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.
• You are required to participate in early state design and feature definition activities.
• Responsible for implementing robust data pipeline using Microsoft toolset and Databricks Stack
• Responsible for creating reusable and scalable data pipelines
• You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.

Skills:

• Advanced working knowledge and experience with relational and non-relational databases
• Experience building and optimizing Big Data pipelines, architectures, and data sets
• Strong analytic skills related to working with unstructured datasets
• Expertise in Azure Data factory, Databricks, EventHub, Python, PySpark, Azure Synapse and SQL.
• Azure Devops experience mandatory to deploy the Data Objects.

Qualifications and Experience:

• 5+ years of experience in the Azure Domain with minimum 4+ years of relevant experience.
• Minimum 3 implementation experience from conceptualization to delivery of analytics capabilities on Azure platform.
• Automotive domain experience is must.
• Knowledge/Experience with Azure cloud stack.
• Knowledge of SAP IS-Auto, SAP BW/BO, Tableau, PowerBI will be considered as added advantage.
• Bachelor’s degree in computers or allied field of study.
• Certification on Azure platform toolsets will be considered as added advantage