Senior Azure Data Engineer - ETL

MUST HAVES:

• Experience with the Microsoft suite of technology including: Azure Data Factory, Azure SQL Database, Azure Data Lake, and Power BI
• Experience with the Amazon Web Services (AWS) suite of technology including: Amazon Aurora, Amazon Relational Database Search, Glue, OpenSearch, etc.
• Experience with developing data extraction, transformation, and load programs (daily and initial load) functionality on a wide range of data repositories (structured and semi-structured files, relational and multi-dimensional data stores)
• Experience with developing, implementing, and maintaining schedule/dependency logic for Extract Transform Load (ETL) scripts
• Experience with data integration, data cleansing, and data analytics
• Experience with data modeling and design principles for data marts and data warehouses
• Experience in database management and administration
• Experience in constructing complex SQL queries and performance tuning
• Experience with Git and knowledge of source control strategies
• Experience creating technical documentation including ETL source-to-target mappings, data model diagrams, architecture artifacts, detailed design documents, etc.
• Experience in eliciting, documenting, and validating business requirements
• Experience in software development lifecycle from Development, to Testing and Quality Assurance, to Delivery
• Solid communications skills: the ability to understand, comprehend, and follow instructions and requirements to communicate clearly and effectively both verbally and in writing.

Nice to have skills:

• Knowledge and understanding of Accessibility for Ontarians with Disability Act (AODA) and related regulations and standards
• Experience designing visual dashboards to highlight trends and Key Performance Indicators (KPIs), and developing canned reports (scheduled and ad-hoc)
• Experience with programming languages such as C#, developing application code such as APIs, Functions, etc.
• Experience writing scripts with PowerShell and other scripting languages

Description

• Hybrid Work Environment: The successful candidate is expected to work onsite at 222 Jarvis St 3 days per week, and may work remotely up to 2 days per week.

Responsibilities:

• Work as an integrated member of the Data Management team to support cross functional data initiatives with various business clients
• Interact with Business teams to better understand the technical functionality requirements and analyze data requirements and their impact to the existing systems
• Responsible in connecting various source and target systems (Oracle, Microsoft and Amazon Aurora, Amazon Relational Database Services, databases, APIs, flat files, etc.) and performing Extract, Transform, and Load (ETL) activities
• Work with team members to develop ETL within Azure Data Factory or Amazon Web Services (AWS) equivalent such as Glue, Kinesis Data Firehose, SageMaker, Data Wrangler, OpenSearch Service, etc. and SQL code to satisfy business requirements
• Create ETL Mappings and develop stored procedures, SQL queries and tune/optimize SQL code the ETL process and the target database for reporting
• Execute unit tests to ensure the accuracy of code changes, and to verify that data integrity and performance requirements are met
• Participate in meetings and scrums, works on incident and enhancement/maintenance tickets assignments and delivers excellent technical support
• Create and maintains technical documents, including data models

Experience and Skill Set Requirements

Responsibilities/Assignment Deliverables Senior ETL Developer

• Designs, develops, and maintains database back-end and ETL applications for Data Warehouse and Business Intelligence solutions. Uses knowledge of ETL languages and development techniques to build and expand application functionality by writing high-quality code using appropriate patterns. Ensures applications remain scalable while complying to standards and following best practices. Tests and debugs applications and ensures that software is properly documented. Collaborates clearly and professionally as part of a development team and communicates with clients.

Evaluation Criteria

Core Technical Skills - 50%

• Experience with the Microsoft suite of technology including: Azure Data Factory, Azure SQL Database, Azure Data Lake, and Power BI
• Experience with the Amazon Web Services (AWS) suite of technology including: Amazon Aurora, Amazon Relational Database Search, Glue, OpenSearch, etc.
• Experience with developing data extraction, transformation, and load programs (daily and initial load) functionality on a wide range of data repositories (structured and semi-structured files, relational and multi-dimensional data stores)
• Experience with developing, implementing, and maintaining schedule/dependency logic for Extract Transform Load (ETL) scripts
• Experience with data integration, data cleansing, and data analytics
• Experience with data modeling and design principles for data marts and data warehouses
• Experience in database management and administration
• Experience in constructing complex SQL queries and performance tuning
• Experience creating technical documentation including ETL source-to-target mappings, data model diagrams, architecture artifacts, detailed design documents, etc.

Other Technical Skills - 35%

• Experience designing visual dashboards to highlight trends and Key Performance Indicators (KPIs), and developing canned reports (scheduled and ad-hoc)
• Experience with programming languages such as C#, developing application code such as APIs, Functions, etc.
• Experience writing scripts with PowerShell and other scripting languages
• Experience with Git and knowledge of source control strategies
• Experience in software development lifecycle from Development, to Testing and Quality Assurance, to Delivery

Communications Skills - 10%

• Experience in eliciting, documenting, and validating business requirements
• Solid communications skills: the ability to understand, comprehend, and follow instructions and requirements to communicate clearly and effectively both verbally and in writing.

Standards and Best Practices - 5%

• Knowledge and understanding of Accessibility for Ontarians with Disability Act (AODA) and related regulations and standards