Cloud Data Platform Engineer

KOUD; São Paulo; Hybrid; BRL 180,000 - 360,000; Full time

Job summary

KOUD is seeking a Cloud Data Platform Engineer to design, implement, and optimize scalable Databricks platforms across Azure and/or AWS. You will configure Unity Catalog and Delta Lake, enforce governance and RBAC, and automate infrastructure with Terraform.

The role focuses on platform reliability, security, and integration with data engineering, analytics, and ML initiatives, in a hybrid on-site environment in São Paulo.

Qualifications

• Bachelor's degree or equivalent practical experience.
• Hands-on Databricks administration and platform engineering.
• Experience with Azure and/or AWS cloud platforms.
• Experience with IaC using Terraform.
• Experience with CI/CD (Azure DevOps / GitHub Actions).
• Knowledge of security controls and RBAC.

Responsibilities

• Design, implement, and administer Databricks workspaces and clusters across Azure/AWS.
• Build scalable, secure, highly available Databricks environments.
• Define platform standards, governance, and operational best practices.
• Configure Unity Catalog, Delta Lake, workspace governance, and IAM controls.
• Plan platform lifecycle management, upgrades, and capacity planning.
• Provision cloud infra using Terraform; automate deployments and configs.
• Design secure networks, private connectivity, and integration patterns.
• Monitor health, performance, and capacity; implement observability.

Skills

Fluent English

Problem-solving

Communication

Independent work

Education

Bachelor's degree or equivalent

Tools

Databricks

Terraform

Azure DevOps

GitHub Actions

Unity Catalog

Delta Lake

Python

PowerShell

Bash

Job description

Koud is looking for a Cloud Data Platform Engineer to work with one of our clients.The position focuses on Databricks platform engineering, cloud infrastructure, automation, security, governance, and operational reliability across Azure and/or AWS environments. The professional will be responsible for designing, implementing, administering, and optimizing scalable and secure data platforms, supporting data engineering, analytics, machine learning, and artificial intelligence initiatives.

Key Responsibilities:

• Design, implement, and administer Databricks workspaces, clusters, and platform components across Azure and/or AWS environments;
• Build and maintain scalable, secure, highly available, and reliable Databricks environments;
• Define platform standards, architectures, governance frameworks, and operational best practices;
• Configure and administer Unity Catalog, Delta Lake, workspace governance, and identity and access controls;
• Implement platform lifecycle management, upgrades, capacity planning, and optimization;
• Provision and manage cloud infrastructure using Infrastructure as Code tools, particularly Terraform;
• Automate platform deployments, configuration management, and operational processes;
• Design secure networks, private connectivity, and integration patterns across cloud environments;
• Implement backup, disaster recovery, high availability, and business continuity solutions;
• Monitor platform health, availability, performance, and capacity;
• Develop observability solutions for infrastructure and data workloads, including monitoring, logging, and auditing;
• Identify and troubleshoot issues related to the platform, networking, infrastructure, and workloads;
• Conduct root cause analyses and implement corrective actions;
• Implement enterprise security controls, RBAC, least-privilege access models, secrets management, encryption, and key management;
• Support compliance requirements and security audits;
• Develop operational runbooks, support processes, and reusable automation standards;
• Work closely with Data Engineering, Data Science, AI Engineering, and Cloud Infrastructure teams;
• Provide technical guidance on platform architecture, operational excellence, and cloud engineering best practices;
• Support the technical development of other professionals and contribute to the continuous evolution of the platform.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related fields, or equivalent practical experience;
• Strong experience in Cloud Engineering, Platform Engineering, Infrastructure Engineering, or related areas;
• Strong hands-on experience implementing and administering Databricks environments;
• Strong experience with enterprise cloud platforms, particularly Azure and/or AWS;
• Experience designing secure, scalable, and highly available cloud solutions;
• Solid knowledge of Databricks administration, including workspace management, cluster configuration and optimization, workflows, and job scheduling;
• Strong hands-on experience with Unity Catalog and Delta Lake;
• Strong experience with Infrastructure as Code, particularly Terraform;
• Experience with CI/CD pipelines using Azure DevOps and/or GitHub Actions;
• Knowledge of Python, PowerShell, and/or Bash for automation;
• Knowledge of cloud networking, including VNET/VPC design and Private Endpoints;
• Experience with RBAC, identity federation, secrets management, encryption, and key management;
• Experience with monitoring and observability tools such as Azure Monitor, CloudWatch, and Log Analytics;
• Strong problem-solving skills and the ability to work independently in complex enterprise environments;
• Good communication skills and the ability to collaborate with different technical teams and stakeholders;
• Fluent English.

Preferred:

• Databricks Certified Platform Administrator certification;
• Databricks Certified Data Engineer Professional certification;
• Azure Administrator Associate or Azure Solutions Architect certification;
• AWS Solutions Architect certification;
• Experience implementing enterprise AI and Machine Learning platforms;
• Experience in regulated environments, particularly in the Insurance or Financial Services sectors;
• Experience with AI and Machine Learning capabilities within the Databricks ecosystem.

Additional Information:

• Hybrid position, with 3 days per week working on-site in Pinheiros, São Paulo.