Data Engineer

Data Engineer - J48368
Knowledge/Qualifications/ Technical Competencies
Expert in knowledge in Azure Data Factory, pipeline creation, Integration Runtimes, triggers, Understanding of ARM Template, capable of handling complex pipeline that involves downloading data from heterogenous sources, parsing complex json files using dataflows and pipelines.
Expert level knowledge in Azure Databricks, Good understanding of Delta Lake, medallion architecture, Autoloader, Delta Live Tables, Delta Sharing, Workflows etc
Expert level knowledge in MS SQL and its fundaments like indexes, CTE, Joins, Views, Functions, Performance Tuning etc
Expert level knowledge in Python/Pyspark programming language for sourcing data from heterogeneous sources, defining strategy for data lakes, data flow, aggregation, summarization for optimizing the performance etc
Should be able to analyse and understand complex data.
Experience in working with relational databases preferably MS SQL.
Design of end-to-end solutions using ADF technology and related components
Good to have reporting knowledge like PowerBI

Job Responsibilities
Resource must have good oral and written communication skills and be able to work both independently and within a team framework
Understand Business requirement and actively provide inputs from Data perspective
Understand the underlying data and flow of data
Build simple to complex pipelines & dataflows in ADF and Databricks
Should be able to implement modules that has security and authorization frameworks
Recognize and adapt to the changes in processes as the project evolves in size and function
Deep understanding of SQL and python
Strong analytical, quantitative, problem solving, and organizational skills
Attention to detail and ability to coordinate multiple tasks, set priorities, and meet deadlines
Strong and effective interpersonal and communication skills and the ability to interact professionally with a diverse group of users and staff

Knowledge/Qualifications/ Competency Desirable
Bachelors Degree in Engineering Computer Science or equivalent experience (Required)

Required Candidate profile

Candidate Experience Should Be : 4 To 7 Candidate Degree Should Be : BCA,BE-Comp/IT,BEd,BE-Other