Lead Data Engineer

Providence India; Hyderabad; On-site; INR 3,200,000 - 5,200,000; Full time

Job summary

The role emphasizes Azure Data Factory, Databricks, data modeling, and delivering reliable reports via Power BI. Strong SQL, Python, and API experience are essential.

Qualifications

• 6-9 years in Azure Data Engineering
• BI Development experience required
• Cloud tech experience: Azure/AWS/Google Cloud
• Apache Spark and Azure Databricks experience
• Microsoft Fabric knowledge beneficial
• Strong problem solving and communication skills
• Agile/sprints experience in global teams
• Bachelor's Degree in CS/Analytics/IS

Responsibilities

• Engineer, develop, and support the lifecycle of data archival and conversion services
• Work with Azure Data Factory, Databricks, and reporting tools to deliver analytics
• Create APIs and governance standards for data analytics
• Collaborate with DWH and reporting teams on data integrity and modeling
• Develop scripts and microservices for data automation

Skills

SQL

Azure SQL

T-SQL

Data warehousing

Databricks

Python

Spark

API development

API governance

Azure services

Power BI

Education

Bachelor's Degree

Tools

Azure Data Factory

Power BI

Azure Databricks

Azure Functions

Job description

Lead Data Engineer, Enterprise Platform and Technology Services - LES Data Team

Organization Background

The ESI Enablement & Platform Technology - LES Team provides analytical services to business units and IS Stakeholders. In this role you will engineer, develop, and support the complete lifecycle of the Enterprise Data Archival and Electronic Data Conversion service, Caregiver Training, Data Warehouse, and a variety of Azure Services for ETL/ELT processes. The successful candidate will possess a strong understanding of Azure Data Services, Data Analysis, and ability to articulate findings and recommendations to stakeholders.

What will you be responsible for?

• Strong knowledge of SQL, Azure SQL, TSQL
• Strong knowledge of database, datawarehouse and data analysis
• Strong knowledge of Databricks, python, spark.
• Working experience in writing scripts on C# or similar languages for API development
• Integrate unstructured data from multiple systems to create a single source of truth
• Develop, monitor, and manage report curation and data modeling
• Creating/implementing/Adopting API Governance and Standards for Data Analytics
• Creating APIs to integrate unstructured data from a variety of systems to create a single source of truth (SailPoint, ServiceNow, Project Online, ADO, Intune, Azure licensing, and other systems of records for single/bi-direction data sets)
• Working on Azure Services like Azure Data Factory, Azure Workbook, Log Analytics, Azure Functions, and other data analytics tools
• Leveraging various Azure Services such as Azure Storage, Logic Apps, Data Factory, Azure SQL SaaS, Runbooks, Automation, Graph API, and various other data tools Developing/maintaining microservice automation (Prisma Cloud/Kubernetes)
• Working experience in data visualization, PowerBI, writing complex SQL queries

What would your day look like?

• Work closely with service teams to determine project engagement and priority, business data reporting and analysis requirements
• Gather Requirements, Data analysis and modelling
• Create scripts to support enhancements
• Work with external vendors on bi-directional API integrations
• Collaborate with DWH and Reporting teams on data analysis and integrity
• Gather requirements, develop, and implement solutions across all endpoint technologies in the Microsoft stack

Who we are looking for?

• Minimum 6-9 years of experience in Azure Data Engineering
• Minimum 6-9 years of experience in BI Development
• 6-9 years of experience working in cloud technologies such as Azure/AWS/Google cloud service
• Experience with Apache Spark and Azure Databricks
• Working knowledge of Microsoft Fabric
• Ability to troubleshoot technical problems, demonstrate problem solving
• Ability to effectively communicate and excellent interpersonal skills
• Experience working in Agile/sprints in a global team setup
• Bachelor's Degree in Computer Science, Data Analytics, Business Management, or IS