Lead/ Senior Azure Data Engineer_ Exp: 8+ Years

Atyeti; Pune District; Hybrid; INR 1,800,000 - 3,200,000; Full time

Job summary

Atyeti in Pune, India seeks a Senior Data Engineer/Lead to direct data engineering projects on Azure. You will architect and operationalize data solutions using Azure Data Factory, Databricks, Data Lake Gen2, and SQL, while guiding a team of engineers to align with architectural principles.

You will migrate on-prem data warehouses to Azure cloud, design pipelines, work with Py-Spark and Azure Functions, and enable Power BI reporting.

Responsibilities

• Lead data engineering projects to build and operationalize data solutions for business using Azure services in combination with custom solutions Azure Data Factory, Azure Data Flows, Azure Databricks, Azure Data Lake Gen 2, Azure SQL etc
• Proven experience on leading a team of data engineers providing technical guidance and ensuring alignment with agreed architectural principles
• Experience in migrating on-premise data warehouses to data platforms on AZURE cloud
• Designing and implementing data engineering, ingestion and transformation functions using ADF, Databricks
• Proficient in Py-Spark
• Experience in building Python based APIs on Azure Function Apps
• Experience on Azure Logic apps
• Experience in Lakehouse/Datawarehouse implementation using modern data platform architecture
• Capacity Planning and Performance Tuning on ADF & Databricks pipelines
• Support data visualization development using Power BI
• Exposure across all the SDLC process, including testing and deployment
• Experience in relational and dimensional modelling, including big data technologies
• Experience in Azure DevOps – Build CI/CD pipelines for ADF, ADLS, Databricks, Azure SQL DB etc
• Experience of working in secured Azure environments using Azure KeyVaults, Service Principals, and Managed Identities
• Good to have knowledge on Apigee (Googles API Management)
• Understanding of data masking, encryption and other practices used in handling sensitive data
• Ability to interact with Business for requirement gathering and query resolutions
• Working on off shore office based development teams, collaborating within a team environment and participating in typical project lifecycle activities such as requirement analysis, testing and release
• Develop Azure Data skills within the team through knowledge sharing sessions, articles, etc.
• Adherence to organisations Risk & Controls requirements
• Should have skills for Stakeholder management, process adherence, planning & documentationss

Job description

Role & responsibilities

• Lead data engineering projects to build and operationalize data solutions for business using Azure services in combination with custom solutions Azure Data Factory, Azure Data Flows, Azure Databricks, Azure Data Lake Gen 2, Azure SQL etc
• Proven experience on leading a team of data engineers providing technical guidance and ensuring alignment with agreed architectural principles
• Experience in migrating on-premise data warehouses to data platforms on AZURE cloud
• Designing and implementing data engineering, ingestion and transformation functions using ADF, Databricks
• Proficient in Py-Spark
• Experience in building Python based APIs on Azure Function Apps
• Experience on Azure Logic apps
• Experience in Lakehouse/Datawarehouse implementation using modern data platform architecture
• Capacity Planning and Performance Tuning on ADF & Databricks pipelines
• Support data visualization development using Power BI
• Exposure across all the SDLC process, including testing and deployment
• Experience in relational and dimensional modelling, including big data technologies
• Experience in Azure DevOps – Build CI/CD pipelines for ADF, ADLS, Databricks, Azure SQL DB etc
• Experience of working in secured Azure environments using Azure KeyVaults, Service Principals, and Managed Identities
• Good to have knowledge on Apigee (Googles API Management)
• Understanding of data masking, encryption and other practices used in handling sensitive data
• Ability to interact with Business for requirement gathering and query resolutions
• Working on off shore office based development teams, collaborating within a team environment and participating in typical project lifecycle activities such as requirement analysis, testing and release
• Develop Azure Data skills within the team through knowledge sharing sessions, articles, etc.
• Adherence to organisations Risk & Controls requirements
• Should have skills for Stakeholder management, process adherence, planning & documentationss