Azure Data Engineer

Wipro; Toronto; On-site; CAD 77,000 - 120,000; Full time

Job summary

Wipro Canada in Toronto is seeking an IT professional with 8+ years SDLC experience and deep Azure data engineering expertise to architect scalable data solutions using Azure Data Factory, Databricks, Synapse, and Snowflake. You will design and implement scalable data pipelines, model data with Azure SQL and Data Lake, and drive near real-time processing with EDW architectures.

The role involves CI/CD under Azure DevOps and delivering Power BI analytics.

Qualifications

• 8+ years in SDLC with Azure data solutions.
• Proven experience in building scalable data pipelines and analytics.
• Strong SQL, data modeling and performance tuning.

Responsibilities

• Design and implement scalable data pipelines using Azure Data Factory, Databricks, Synapse and Snowflake.
• Develop data models and storage solutions with Azure SQL and Data Lake.
• Create near real-time data pipelines and EDWs with audit controls.
• Lead CI/CD automation for data solutions using Azure DevOps.
• Build dashboards and reports in Power BI.

Skills

Azure stack

Azure Data Factory

Azure Databricks

Snowflake

Synapse

Power BI

Pyspark

SparkSQL

CI/CD

Azure DevOps

Data Lake

SQL

Tools

Azure Synapse Analytics

Snowflake

Azure Data Lake

Azure Blob Storage

SQL Server

Job description

• IT Professional 8+ years of comprehensive experience in the Software Development Life Cycle (SDLC), specializing in Azure stack with a proven track record of leveraging Microsoft Azure/Cloud Services to architect and implement scalable data solutions. Skilled in utilizing SQL Data Warehouse, Azure SQL Server, Snowflake, Azure Data Lake, Azure Blob Storage, Azure Databricks, and Azure Data Factory for efficient data processing, storage, and analytics (power BI)
• Design and implement scalable data pipelines using Azure Data Factory, Azure Databricks, Synapse and Snowflake Analytics.
• Develop data models and optimized data storage solutions using Azure SQL Database, and Azure Data Lake (Blob Storage).
• Experience in Azure fabric frame work (creating lake house, ware houses)
• Experience in working with One lake and delta lake for fabric workspaces.
• Experience in migration of on-premises data warehouse to Snowflake, Azure Synapse Analytics, enhancing data accessibility and reporting capabilities.
• Experience in designing and developing scalable data processing frameworks using Apache Spark, enabling efficient processing and analysis of large datasets in both batch and streaming modes.
• Proficiency in Pyspark, SparkSQL for developing notebooks in Databricks
• Experience in Spark SQL for developing notebooks in synapse.
• Have Worked on Event Hubs for processing streaming data in to Databricks.
• Experience in loading/writing data in to Azure data lake using ADF, Databricks, Synapse Analytics
• Proficiency in writing complex SQL queries and stored procedures. Proficient in optimizing data-base performance, ensuring efficient data retrieval, and implementing robust data manipulation logic.
• Proficiency in creating near real-time data pipelines, Enterprise Data Warehouses (EDW), DataMart’s, and audit and control frameworks. Skilled in designing and implementing solutions that ensure data integrity, compliance, and efficient reporting.
• Implemented CI/CD pipelines using Azure DevOps, including automated build, test, and deploy-ment processes.
• Designed and implemented a comprehensive framework of controls for the data governance team, ensuring data adheres to standards of accuracy, completeness, consistency, and compliance.
• Led the design and development of robust data loading frameworks and products, enabling efficient and reliable data ingestion, transformation, and integration across multiple platform

Job Description

• IT Professional 8+ years of comprehensive experience in the Software Development Life Cycle (SDLC), specializing in Azure stack with a proven track record of leveraging Microsoft Azure/Cloud Services to architect and implement scalable data solutions. Skilled in utilizing SQL Data Warehouse, Azure SQL Server, Snowflake, Azure Data Lake, Azure Blob Storage, Azure Databricks, and Azure Data Factory for efficient data processing, storage, and analytics (power BI)
• Design and implement scalable data pipelines using Azure Data Factory, Azure Databricks, Synapse and Snowflake Analytics.
• Develop data models and optimized data storage solutions using Azure SQL Database, and Azure Data Lake (Blob Storage).
• Experience in Azure fabric frame work (creating lake house, ware houses)
• Experience in working with One lake and delta lake for fabric workspaces.
• Experience in migration of on-premises data warehouse to Snowflake, Azure Synapse Analytics, enhancing data accessibility and reporting capabilities.
• Experience in designing and developing scalable data processing frameworks using Apache Spark, enabling efficient processing and analysis of large datasets in both batch and streaming modes.
• Proficiency in Pyspark, SparkSQL for developing notebooks in Databricks
• Experience in Spark SQL for developing notebooks in synapse.
• Have Worked on Event Hubs for processing streaming data in to Databricks.
• Experience in loading/writing data in to Azure data lake using ADF, Databricks, Synapse Analytics
• Proficiency in writing complex SQL queries and stored procedures. Proficient in optimizing data-base performance, ensuring efficient data retrieval, and implementing robust data manipulation logic.
• Proficiency in creating near real-time data pipelines, Enterprise Data Warehouses (EDW), DataMart’s, and audit and control frameworks. Skilled in designing and implementing solutions that ensure data integrity, compliance, and efficient reporting.
• Implemented CI/CD pipelines using Azure DevOps, including automated build, test, and deploy-ment processes.
• Designed and implemented a comprehensive framework of controls for the data governance team, ensuring data adheres to standards of accuracy, completeness, consistency, and compliance.
• Led the design and development of robust data loading frameworks and products, enabling efficient and reliable data ingestion, transformation, and integration across multiple platform

Design and develop dashboards, reports in Power BI tool

Mandatory Skills: DataBricks - Data Engineering .

Experience: 5-8 Years

The expected compensation for this role ranges from CAD 77000 to CAD 120000 .

Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with the boldest ambitions. To realize them, we need people inspired by reinvention. Of yourself, your career, and your skills. We want to see the constant evolution of our business and our industry. It has always been in our DNA - as the world around us changes, so do we. Join a business powered by purpose and a place that empowers you to design your own reinvention.

Wipro is committed to creating an accessible, supportive, and inclusive workplace. Reasonable accommodation will be provided to all applicants including persons with disabilities, throughout the recruitment and selection process. Accommodations must be communicated in advance of the application, where possible, and will be reviewed on an individual basis. Wipro provides equal opportunities to all and values diversity.