Senior DataOps/Cloud Data Engineer

Cynet Systems Inc; Toronto; On-site; CAD 80,000 - 120,000; Full time

Job summary

An innovative firm is seeking a skilled Data Engineer to design and optimize data pipelines and database management solutions. This role involves working with cutting-edge Microsoft Azure technologies to develop and maintain efficient data systems. You will be responsible for creating CI/CD pipelines, managing data in cloud environments, and ensuring data integrity across various platforms. Join a forward-thinking team where your expertise in data operations and cloud solutions will make a significant impact on the organization's data strategy. If you are passionate about data management and eager to contribute to transformative projects, this opportunity is for you.

Qualifications

• Expert experience in data/database administration across various platforms.
• Strong knowledge of CI/CD and data provisioning automation.

Responsibilities

• Design and implement data loading, conversion, and migration programs.
• Optimize performance of databases and ensure effective knowledge transfer.

Skills

Cloud DataOPS

Microsoft Azure technologies

Data pipeline development

Data management

Data analysis

Agile project development

Education

Bachelor's degree in Computer Science or related field

Tools

Sybase Power Designer

Azure DevOps

Oracle

Microsoft Azure SQL Database

SQL Server

Job description

Job Description:

• Participate in the review of project, work request, and maintenance release artifacts such as Use Case, System Requirement Specification, etc.
• Deliver detailed system design documents and other supporting documentation conforming to Ministry standards.
• Design, develop, and optimize data/database CI/CD pipelines.
• Design logical and physical data models using Sybase Power Designer.
• Prepare database changes in DACPAC and develop Azure CI/CD release pipelines to promote DACPAC.
• Promote database changes through Development, Integration Testing, QA, UAT, and Production environments.
• Provision data in cloud environments according to requirements.
• Design, develop, and implement data loading, data conversion, and data migration programs.
• Design, develop, and implement data conversions for Very Large Data (VLD) in OLAP and OLTP environments to Cloud SaaS, PaaS, and IaaS platforms.
• Design, develop, and implement fact/dimension models, data mapping, data warehouses, data lakes, and data lakehouses.
• Plan, configure, maintain, and tune performance of Oracle, Microsoft Azure SQL Database, and Synapse databases.
• Support cloud-based heterogeneous data replication solutions.
• Develop complex Oracle PL/SQL programs and SQL Server stored procedures to fulfill project, work request, and maintenance requirements.
• Optimize performance of PL/SQL programs and SQL Server stored procedures.
• Monitor and analyze incidents, providing timely resolutions.
• Perform reference data administration as required.
• Build and test database backup/recovery scripts and automation scripts.
• Perform database administration tasks, including patching/upgrading, monitoring, performance tuning, partitioning, capacity planning, OEM upgrades/enhancements, and achieving.
• Document and resolve any database problems or issues.
• Provide effective knowledge transfer to Ministry staff at all stages of the assignment.

Experience and Skill Set Requirements:

• Cloud DataOPS - 40%
• Expertise with Microsoft Azure technologies, including Azure pipelines, DACPAC, storage accounts, disks, file shares, Azure Key Vault, Azure portal, virtual machines, Azure DevOps, Data Lake, Data Lakehouse, and Azure monitoring tools.
• Experience with cloud data platforms, data management, and data exchange tools and technologies.
• Experience with data pipeline and workflow development, orchestration, deployment, and automation.
• Knowledge of DataOPS principles, best practices, and implementation, along with Agile project development and deployment.
• Experience in CI/CD and data provisioning automation.
• Experience with digital products, data analysis, data exchange, data provisioning, and data security.
• Experience working with structured, semi-structured, and unstructured data collection, ingestion, provisioning, and exchange technologies.
• Experience with DataOPS performance monitoring and tuning.

Database Management - 40%:

• Expert experience in data/database administration, e.g., Microsoft Azure SQL Database, Synapse, Oracle, IMS, DB2, etc.
• Extensive experience with SQL Server Stored Procedures, Oracle PL/SQL, PL/1 development, and performance tuning.

Data Replication Pipelines - 20%:

• Experience with Azure SQL, Oracle, IMS/DB2 data replication.