DevOps Engineer - Azure Databricks

SilverCreek; Edmonton; On-site; CAD 110,000 - 150,000; Full time

Job summary

Silver Creek Software Ltd. in Alberta is seeking a hands-on DevOps Engineer with strong Microsoft Azure and Azure Databricks experience to design, automate, and support reliable deployment and release processes across enterprise environments.

This role focuses on CI/CD, infrastructure as code, environment promotion, identity management, and deployment governance for Databricks and related Azure workloads. You will collaborate with developers, data engineers, security, and business stakeholders.

Qualifications

• Proven DevOps experience with Azure Databricks in cloud or platform engineering.
• Strong CI/CD design using GitHub Actions or Azure DevOps for automated builds and releases.
• Infrastructure-as-code with Terraform and/or Bicep including secrets management.
• Experience promoting changes across Development, Test/UAT, and Production with rollback procedures.
• Solid Git workflows: branching, PRs, reviews, tagging, versioning.

Responsibilities

• Design, implement, and support CI/CD and deployment automation for Azure Databricks.
• Build and maintain automated promotion processes across environments.
• Create and maintain CI/CD pipelines with build, test, and release stages.
• Manage environment configurations, secrets, and deployment parameters.
• Implement secure access patterns using Key Vault and RBAC.
• Monitor deployments, troubleshoot issues, and perform root-cause analysis.

Skills

Azure Databricks

CI/CD

Terraform

GitHub Actions

Azure DevOps

Python

Education

Bachelor's degree in Computer Science or equivalent

Tools

Databricks CLI

Databricks Asset Bundles

Azure CLI

Terraform / Bicep tooling

Job description

Silver Creek Software Ltd. is an Alberta-based consulting firm specializing in data management, analytics, application development, automation, and modern workplace solutions. We partner with public sector and enterprise clients to deliver complex, high-impact technology initiatives. Our team is known for its professionalism, technical depth, and commitment to excellence. We maintain a high standard of hiring and are proud to work alongside driven, capable professionals who consistently go above and beyond for our clients.

We are seeking a hands-on DevOps Engineer with strong Microsoft Azure and Azure Databricks experience to design, automate, and support reliable deployment and release processes across enterprise environments. This role is focused on CI/CD, infrastructure as code, environment promotion, identity and access management, deployment governance, and operational support for Databricks and related Azure workloads.

The successful candidate will work closely with developers, data engineers, architects, security teams, platform administrators, and business stakeholders to establish repeatable, secure, and auditable delivery practices across Development, Test/UAT, and Production environments. A strong software engineering foundation and the ability to troubleshoot application, platform, and deployment issues are important to success in this role.

This opportunity is suitable for both full-time employment or contract arrangements.

Key Responsibilities

• Design, implement, and support CI/CD and deployment automation for Azure Databricks and related Azure services.
• Build and maintain automated promotion processes across Development, Test/UAT, and Production environments.
• Create and maintain CI/CD pipelines using GitHub Actions and/or Azure DevOps, including build, test, validation, approval, and release stages.
• Implement source-control and release-management practices using Git, branching strategies, pull requests, code reviews, tagging, and release versioning.
• Automate Databricks deployments using Databricks Asset Bundles / Declarative Automation Bundles, Databricks CLI, REST APIs, Jobs, Workflows, notebooks, SQL assets, and related tooling.
• Manage environment‑specific configurations, variables, secrets, deployment parameters, and secure application settings across multiple environments.
• Implement and support secure access patterns using Azure Key Vault, service principals, managed identities, Microsoft Entra ID, and role‑based access control (RBAC).
• Support Unity Catalog security and environment isolation, including permissions, catalogs, schemas, and storage credentials.
• Provision and manage repeatable cloud and platform infrastructure using Terraform and/or Bicep.
• Implement automated testing, deployment validation, approval gates, audit history, release controls, and rollback procedures.
• Monitor deployments and platform health, troubleshoot failed releases and production issues, and perform root‑cause analysis.
• Develop and maintain automation and operational scripts using Python, PowerShell or Bash, YAML, JSON, and SQL.
• Prepare deployment runbooks, release documentation, technical standards, and knowledge‑transfer materials for delivery and support teams.
• Collaborate with technical and business stakeholders to improve development, release, security, and operational practices across the solution lifecycle.

Qualifications

• Proven DevOps experience with Azure Databricks in a cloud or platform engineering role, with responsibility for CI/CD, infrastructure automation, release management, and production environments.
• Strong experience designing and maintaining CI/CD pipelines using GitHub Actions and/or Azure DevOps, including automated build, testing, deployment, validation, and approval workflows.
• Infrastructure‑as‑code experience using Terraform and/or Bicep, including management of environment‑specific configurations, variables, secrets, and deployment parameters.
• Experience promoting application, platform, or data changes across Development, Test/UAT, and Production using structured release controls, auditability, and rollback procedures.
• Strong Git experience, including branching strategies, pull requests, code reviews, tagging, and release versioning.
• Experience monitoring deployments, troubleshooting production issues, and performing root‑cause analysis across cloud, application, and platform components.
• Hands‑on experience with Azure Databricks deployment automation and operational support.
• Experience with Databricks Asset Bundles / Declarative Automation Bundles, Databricks CLI, REST APIs, Jobs, Workflows, notebooks, and SQL assets.
• Strong understanding of Azure security and identity services, including Key Vault, Microsoft Entra ID, service principals, managed identities, and RBAC.
• Experience with Unity Catalog permissions and governance, including catalogs, schemas, storage credentials, and environment isolation.
• Scripting and automation skills using Python, PowerShell or Bash, with working knowledge of YAML, JSON, and SQL.
• Strong documentation and communication skills, including deployment runbooks, release documentation, knowledge transfer, and collaboration with developers, architects, security teams, platform administrators, and business stakeholders.
• Bachelor's degree in computer science, engineering, information technology, or a related discipline, or an equivalent combination of education and professional experience.
• Experience working in enterprise, government, public sector, financial services, utilities, or other regulated environments.
• Relevant Microsoft Azure and/or Databricks certifications.
• Experience supporting Azure data services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Azure SQL.
• Familiarity with modern data engineering and lakehouse concepts, including Delta Lake, medallion architecture, ETL/ELT pipelines, and Databricks data workloads.
• Software development experience with .NET/C#, APIs, web applications, or other enterprise application technologies.
• Experience with application observability, monitoring, logging, and production support using Azure Monitor, Application Insights, Log Analytics, or similar tools.
• Experience modernizing manual or legacy deployment processes into governed, automated, and repeatable delivery workflows.
• Consulting or client‑facing experience.
• Experience working within Agile or iterative delivery environments.

Join Us

Silver Creek offers the opportunity to work with a high-performing team on meaningful projects that deliver real value to our clients. We provide flexibility, autonomy, and trust, while holding ourselves to a high standard of professionalism and accountability.