Lead Data Engineer - Azure

Blend; Hyderabad; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Blend is seeking a Lead Data Engineer to enhance our data engineering teams. You will architect new projects and select the tech stack while ensuring proper data stewardship. Lead a dynamic team, where your expertise will drive significant initiatives in data governance and data quality, while nurturing junior engineers.

Qualifications

• 7+ years experience as a data engineer or similar role.
• Experience executing end-to-end data projects.
• Strong SQL skills and understanding of relational databases.

Responsibilities

• Lead and mentor data engineering teams.
• Create and maintain data systems, pipelines, and tools.
• Drive development efforts for high-quality data solutions.

Skills

Data modeling

Data mining

Segmentation techniques

SQL

Spark

PySpark

Distributed processing

Education

Degree in Computer Science, Data Science, Mathematics, IT

Tools

Azure Synapse

Data Factory (ADF)

Docker

Kubernetes

Job description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visitwww.blend360.com

Job Description

As a Lead Data Engineer, your role is to spearhead the data engineering teams and elevate the team to the next level! You will be responsible for laying out the architecture of the new project as well as selecting the tech stack associated with it. You will plan out the development cycles deploying AGILE if possible and create the foundations for good data stewardship with our new data products!

You will also set up a solid code framework that needs to be built to purpose yet have enough flexibility to adapt to new business use cases tough but rewarding challenge!

Responsibilities

• Collaborate with several stakeholders to deeply understand the needs of data practitioners to deliver at scale
• Lead Data Engineers to define, build and maintain Data Platform
• Work on building Data Lake in Azure Fabric processing data from multiple sources
• Migrating existing data store from Azure Synapse to Azure Fabric
• Implement data governance and access control
• Drive development effort End-to-End for on-time delivery of high-quality solutions that conform torequirements, conform to the architectural vision, and comply with all applicable standards.
• Present technical solutions, capabilities, considerations, and features in business terms. Effectively communicate status, issues, and risks in a precise and timely manner.
• Further develop critical initiatives, such as Data Discovery, Data Lineage and Data Quality
• Leading team and Mentor junior resources
• Help your team members grow in their role and achieve their career aspirations
• Build data systems, pipelines, analytical tools and programs
• Conduct complex data analysis and report on results

Qualifications

• 7+ Years of Experience as a data engineer or similar role in Azure Synapses, ADF or
• Degree in Computer Science, Data Science, Mathematics, IT, or similar field
• Must have experience executing projects end to end.
• At least one data engineering project should have worked in Azure Synapse, ADF or
• Should be experienced in handling multiple data sources
• Technical expertise with data models, data mining, and segmentation techniques
• Deep understanding, both conceptually and in practice of at least one object orientated
• library (Python, pySpark)
• Strong SQL skills and a good understanding of existing SQL warehouses and relational
• databases.
• Strong Spark, PySpark, Spark SQL skills and good understanding of distributed
• processing frameworks.
• Build large-scale batches and real-time data pipelines.
• Ability to work independently and mentor junior resources.
• Desire to lead and develop a team of Data Engineers across multiple levels
• Experience or knowledge in Should have experience or knowledge in Data Governance
• Azure Cloud experience with Data modeling, CI\CD, Agile Methodologies,Docker\Kubernetes