RQ08590 - Software Developer - ETL - Senior

Rubicon Path; Toronto; On-site; CAD 90,000 - 120,000; Full time

Job summary

A leading company is seeking a Senior Software Developer specializing in ETL processes to design and implement data pipelines. The ideal candidate will have extensive experience with Azure Data Factory, Informatica, and Oracle PL/SQL, ensuring high-quality data transformations and performance tuning across various environments.

Qualifications

• Extensive experience in requirements gathering and business process understanding.
• Experience with Azure Data Factory, including performance tuning and complex data transformations.
• Extensive experience designing and developing SQL Server Stored Procedures and Oracle PL/SQL programs.

Responsibilities

• Participate in project reviews and deliver detailed system design documents.
• Implement data pipelines using Azure Data Factory and Informatica.
• Monitor application functionality and performance daily.

Skills

ETL Development

Data Integration

Azure Data Factory

Oracle PL/SQL

Performance Tuning

Business Intelligence

CI/CD

Tools

Informatica

Databricks

IBM Cognos

Microsoft PowerBI

Job description

RQ08590 - Software Developer - ETL - Senior

About the job RQ08590 - Software Developer - ETL - Senior

Scope of Services and Deliverables - The Services and Deliverables to be provided by the Vendor will include the following:

• Participate in the review of project, work request, and maintenance release artefacts such as Use Case, System Requirement Specification, etc.
• Deliver detailed system design documents and supporting documentation conforming to Ministry standards.
• Design logical and physical data models using Sybase Power Designer.
• Implement data pipelines using Azure Data Factory (ADF), stored procedures, and Informatica PowerCenter for extraction, transformation, and loading activities.
• Implement solutions using Logic App and Function App.
• Generate structured JSON files using ADF/SQL.
• Develop Azure CI/CD pipelines to automate ADF releases.
• Develop complex Oracle PL/SQL programs to meet project requirements.
• Implement complex data conversions, e.g., binary to character, EBCDIC to UTF.
• Implement complex data transformations for derived and calculated values.
• Integrate data sets from diverse source systems.
• Implement ADF for initial data load and incremental load.
• Promote ADF and Informatica ETL/ELT processes across all Ministry environments including Development, Integration Testing, QA, UAT, and Production.
• Resolve and troubleshoot technical issues in ADF pipelines, stored procedures, and Informatica workflows.
• Monitor and analyze incidents, providing timely resolutions.
• Collaborate with IT professionals throughout the Software Development Life Cycle.
• Document information from business stakeholders and subject matter experts.
• Optimize performance of ADF pipelines, Azure SQL, Synapse databases, and Informatica workflows.
• Monitor application functionality and performance daily.
• Provide effective knowledge transfer to Ministry staff at all stages of the assignment.

Skills, Experience, and Skill Set Requirements

Evaluation Table

ETL Development - Data Integration 40%

• Extensive experience in requirements gathering and business process understanding to design high-quality data transformations.
• Experience in design, development, and implementation with Azure Data Factory, Databricks, including:
• Self-Hosted Integration Runtime
• Performance tuning on large data volumes
• Complex data transformations
• ADF Debugger for data validation
• Experience with Informatica tools such as Workflow Manager, Repository Manager, Designer, and Workflow Monitor.
• Experience developing solutions using Logic App and Function App.
• Experience with Rapid Application Development (RAD) methodologies.
• Experience maintaining and improving existing ETL processes.
• Experience investigating data issues within ETL pipelines.
• Experience with Business Intelligence tools like IBM Cognos and Microsoft PowerBI.
• Experience with CI/CD (DevOps) pipelines, including branching, pull requests, build, release, and managing multiple repositories.

Oracle PL/SQL Development 10%

• Extensive experience designing and developing SQL Server Stored Procedures and Oracle PL/SQL programs.

Database Technologies 10%

• Experience with Oracle, Microsoft SQL Server, and related tools.
• Knowledge of relational and hierarchical database technologies.
• Extensive knowledge of Azure SQL Database and Synapse platforms, including performance tuning on large data volumes.

Must haves:

• Extensive experience in requirements gathering and business process understanding to design correct and high-quality data transformations.