Technology Lead - Azure and Databricks Admin

Infosys; Calgary; On-site; CAD 120,000 - 180,000; Full time

Job summary

Infosys is seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The role covers workspace administration, cluster management, governance, cloud integrations, and platform monitoring.

Ideal candidates will have strong Azure Databricks expertise (preferred), with exposure to AWS and GCP Databricks environments, and will support multi-region deployments and automation using Terraform, Azure

Qualifications

• Bachelor’s degree or foreign equivalent required; three years of related experience may substitute for education.
• 4+ years IT experience with Databricks administration or cloud platform stewardship.

Responsibilities

• Administer and manage Databricks workspaces across development, testing, and production environments.
• Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
• Manage Unity Catalog implementation, governance, and access controls.
• Perform platform upgrades, maintenance, and health checks.
• Support multi-workspace and multi-region deployments.
• Create, configure, and manage all-purpose and job clusters.
• Implement cluster policies to control costs and ensure compliance.
• Monitor cluster utilization, performance, and cost efficiency.
• Troubleshoot cluster startup, termination, and execution issues.
• Automate Databricks administration using Terraform, Azure DevOps, GitHub Actions, Databricks CLI, REST APIs.
• Manage CI/CD pipelines for Databricks deployments and IaC initiatives.
• Participate in incident, problem, and change management activities.
• Collaborate with engineering, infrastructure, security, and application teams.

Skills

Analytical thinking

Troubleshooting

Excellent communication

Stakeholder coordination

Documentation mindset

Global delivery awareness

Education

Bachelor's degree or foreign equivalent in IT or related field

Tools

Terraform

Azure DevOps

GitHub Actions

Databricks CLI

REST APIs

Power BI

Azure AD

AWS IAM

Job description

We are seeking an experienced Databricks Platform Administrator to manage, secure, optimize, and support enterprise Databricks environments across cloud platforms. The ideal candidate will be responsible for workspace administration, cluster management, security governance, cloud integrations, platform monitoring, automation, and operational support. This role requires strong expertise in Azure Databricks (preferred), along with exposure to AWS and GCP Databricks environments.

Required Qualifications

• Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply.
• Candidate must be located within commuting distance of Calgary, Alberta or be willing to relocate to the areas.
• Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience.

Key Responsibilities

Platform Administration

• Administer and manage Databricks workspaces across development, test, and production environments.
• Configure and maintain Databricks workspaces, metastore, catalogs, clusters, pools, and policies.
• Manage Unity Catalog implementation, governance, and access controls.
• Perform platform upgrades, maintenance, and health checks.
• Support multi-workspace and multi-region deployments.

Cluster Management

• Create, configure, and manage all-purpose and job clusters.
• Implement cluster policies to control costs and ensure compliance.
• Optimize cluster sizing and auto-scaling configurations.
• Monitor cluster utilization, performance, and cost efficiency.
• Troubleshoot cluster startup, termination, and execution issues.

Security & Governance

• Implement RBAC and workspace access controls.
• Integrate Databricks with Azure AD, AWS IAM, or GCP Identity services.
• Manage service principals, managed identities, and secret scopes.
• Implement Unity Catalog governance and data security policies.
• Support audit logging, compliance, and platform security reviews.

Cloud Integration

• Integrate Databricks with:
  • Azure Data Lake Storage (ADLS)
  • AWS S3
  • Azure Key Vault
  • AWS Secrets Manager
  • SQL Databases
  • Event Hub
  • Kafka
  • Snowflake
  • Power BI

Monitoring & Performance

• Monitor platform health and resource utilization.
• Establish proactive alerting and monitoring mechanisms.
• Analyze Spark job performance and recommend optimizations.
• Troubleshoot platform incidents and performance bottlenecks.
• Support SLA-driven production environments.

DevOps & Automation

• Automate Databricks administration using:
  • Terraform
  • Azure DevOps
  • GitHub Actions
  • Databricks CLI
  • REST APIs
• Manage CI/CD pipelines for Databricks deployments.
• Support Infrastructure as Code (IaC) initiatives.

Operational Support

• Participate in incident, problem, and change management activities.
• Support patching, upgrades, certificate renewals, and maintenance activities.
• Create and maintain SOPs, runbooks, and technical documentation.
• Collaborate with engineering, infrastructure, security, and application teams.

Required Skills

Databricks

• Databricks Workspace Administration
• Unity Catalog
• Cluster Management
• Cluster Policies
• Instance Pools
• Jobs & Workflows
• SQL Warehouses
• Databricks CLI
• REST APIs
• Delta Lake

Cloud Platforms

• Azure Databricks (Preferred)
• AWS Databricks
• GCP Databricks

Preferred Skill And Experience

• Experience with enterprise-scale Azure Databricks environments.
• Strong understanding of data lake architecture and governance frameworks.
• Experience supporting production environments with strict SLA requirements.

Soft Skills

• Strong analytical and troubleshooting skills.
• Excellent communication and stakeholder coordination.
• Ability to work in a global delivery model.
• Documentation and knowledge sharing mindset.

Experience

• 8-10 years of experience in IT Industry
• 5+ years of Databricks Administration experience.
• Prior experience in large enterprise or managed services engagements preferred.
• Exposure to Infosys‑scale delivery environments is an advantage.

The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements.

EEO

Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.