Senior Databricks Consultant

EY; Hyderabad; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

EY GDS Data and Analytics Azure Data Engineer with Databricks is seeking candidates with strong technology and data understanding in big data engineering. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team.

Your key responsibilities include developing and deploying big data pipelines in Azure, ETL design and cloud migration, interacting with senior leaders to align with business goals, and designing/optimizing code for performance.

Qualifications

• 4–8 years of IT experience with computer science background.
• Experience in Agile delivery methodologies.
• Excellent communication and ownership in problem resolution.
• Coordinate with stakeholders and implement requirements using ADF.

Responsibilities

• Develop & deploy big data pipelines in a cloud environment using Azure Cloud services
• ETL design, development and migration of existing on-prem ETL routines to Cloud Service
• Interact with senior leaders, understand their business goals, contribute to the delivery of the workstreams
• Design and optimize model codes for faster execution

Skills

Azure Data Factory

Databricks

PySpark

Python

SQL Server

Azure Functions

Azure Synapse

Azure Data Lake/Blob

Agile methodologies

Education

Bachelor's degree in Computer Science

Job description

Job Summary

EY GDS Data and Analytics (D&A) Azure Data Engineer with Databricks

Were looking for candidates with strong technology and data understanding in big data engineering space, having proven delivery capability. This is a fantastic opportunity to be part of a leading firm as well as a part of a growing Data and Analytics team.

Your key responsibilities

• Develop & deploy big data pipelines in a cloud environment using Azure Cloud services
• ETL design, development and migration of existing on-prem ETL routines to Cloud Service
• Interact with senior leaders, understand their business goals, contribute to the delivery of the workstreams
• Design and optimize model codes for faster execution

Skills and attributes for success

• Overall 4+ years of IT experience with 2+ Relevant experience in Azure Data Factory (ADF) and good hands-on with Exposure to latest ADF Version.
• Sound knowledge in DataBricks, Pyspark, Python & SQL Server
• Hands-on experience on Azure functions & Azure synapse (Formerly SQL Data Warehouse)
• Should have project experience in Azure Data Lake / Blob (Storage purpose)
• Should have basic understanding on Batch Account configuration, various control options
• Should be able to coordinate independently with business stake holders and understand the business requirements, implement the requirements using ADF

To qualify for the role, you must have

• Be a computer science graduate or equivalent with 4-8 years of industry experience
• Have working experience in an Agile base delivery methodology (Preferable)
• Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution.
• Excellent communicator (written and verbal formal and informal).
• Participate in all aspects of Big Data solution delivery life cycle including analysis, design, development, testing, production deployment, and support.

Ideally, you'll also have

• Client management skills

What we look for

• People with technical experience and enthusiasm to learn new things in this fast-moving environment