Cloud Data Platform Engineer (Databricks Platform Engineering) (4813)

Keep Simple; São Paulo; Hybrid; BRL 240,000 - 360,000; Full time

Job summary

Keep Simple is seeking a Cloud Data Platform Engineer to design, deploy, administer, and optimize enterprise-scale Databricks platforms across cloud environments. You will build secure, scalable data platforms enabling analytics, ML, and AI-driven solutions.

The ideal candidate has 5+ years in cloud/platform engineering, hands-on Databricks administration, and experience with Azure and/or AWS, Unity Catalog, Delta Lake, and IaC tooling. Hybrid work option in São Paulo region.

Qualifications

• Bachelor's degree in CS/IT/Engineering or related field.
• 5+ years of experience in cloud/platform/infrastructure engineering.
• 3+ years deploying and administering Databricks.
• Experience supporting enterprise cloud platforms in Azure and/or AWS.
• Experience designing highly available, secure, and scalable cloud solutions.

Responsibilities

• Design and deploy Databricks workspaces, clusters, and platform components in Azure and/or AWS.
• Build and maintain secure, scalable, and highly available Databricks environments.
• Configure Unity Catalog, Delta Lake, and workspace governance frameworks.
• Implement platform lifecycle management, upgrades, and capacity planning.
• Provision infrastructure using IaC tools such as Terraform.
• Automate platform deployments and configuration management.
• Design networking, private connectivity, and secure integration patterns.
• Implement backup, disaster recovery, and high-availability solutions.
• Monitor platform health, availability, and performance.
• Collaborate with Data Engineers, Data Scientists, AI Engineers, and Cloud Infra teams.

Skills

Databricks Administration

Unity Catalog

Delta Lake

Workspace Management

Cluster Configuration

Databricks Workflows

Job Scheduling

Identity and Access Management

Education

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field

Tools

Terraform

Azure DevOps

GitHub Actions

CI/CD Pipelines

PowerShell

Python

Bash scripting

Job description

Job description

Come work for a large global financial and insurance products company! This is your chance!!

Start a successful career in a renowned company in the international market! Great opportunity!

Global insurance and asset management company seeks a responsible, organized, dynamic and team-oriented person.

Main responsibilities

We are seeking a highly skilled Cloud Data Platform Engineer to design, deploy, administer, and optimizeenterprise-scale Databricks platforms across cloud environments. This role will be responsible for building andmaintaining secure, scalable, and reliable data platforms that enable analytics, machine learning, and AI-drivenbusiness solutions.

The ideal candidate combines expertise in cloud infrastructure, platform engineering, data technologies,security, automation, and Databricks administration. Experience with Databricks AI capabilities, includingGenerative AI, Vector Search, MLflow, and Mosaic AI, is highly desirable.

Key Responsibilities

Databricks Platform Engineering:

• Design and deploy Databricks workspaces, clusters, and platform components in Azure and/or AWS;
• Build and maintain scalable, secure, and highly available Databricks environments;
• Establish platform standards, architecture patterns, and operational best practices;
• Configure and manage Unity Catalog, Delta Lake, and workspace governance frameworks;
• Implement platform lifecycle management, upgrades, and capacity planning.

Cloud Infrastructure & Automation:

• Provision infrastructure using Infrastructure-as-Code tools such as Terraform;
• Automate platform deployments and configuration management;
• Design networking, private connectivity, and secure integration patterns;
• Implement backup, disaster recovery, and high-availability solutions;
• Optimize cloud resource utilization and manage platform costs.

Security & Governance:

• Implement enterprise security controls and compliance requirements;
• Configure role-based access controls and least-privilege access models;
• Manage secrets, key vault integrations, and encryption standards;
• Establish monitoring, audit logging, and governance controls;
• Support regulatory and security audits.

Platform Operations & Reliability:

• Monitor platform health, availability, and performance;
• Develop observability solutions for infrastructure and data workloads;
• Troubleshoot platform, networking, and workload issues;
• Create operational runbooks and support processes;
• Lead root-cause analysis and remediation efforts.

Collaboration & Technical Leadership:

• Partner with Data Engineers, Data Scientists, AI Engineers, and Cloud Infrastructure teams;
• Provide technical guidance on platform architecture and operational excellence;
• Develop engineering standards and reusable automation patterns;
• Mentor engineers and promote cloud engineering best practices.

AI & Advanced Analytics Support (Preferred)

• Support Databricks AI and machine learning environments;
• Enable MLflow experimentation, model tracking, and deployment capabilities;
• Support Mosaic AI, Vector Search, Feature Store, and RAG architectures;
• Assist teams implementing Generative AI and Large Language Model solutions;
• Develop infrastructure and governance frameworks supporting AI workloads;
• Evaluate new Databricks AI capabilities and recommend adoption strategies.

Requirements and skills

Required Qualifications

Education:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

Experience:

• 5+ years of experience in cloud engineering, platform engineering, or infrastructure engineering;
• 3+ years of hands‑on experience deploying and administering Databricks environments;
• Experience supporting enterprise cloud platforms in Azure and/or AWS;
• Experience designing highly available, secure, and scalable cloud solutions.

Technical Skills

Databricks:

• Databricks Administration;
• Unity Catalog;
• Delta Lake;
• Workspace Management;
• Cluster Configuration and Optimization;
• Databricks Workflows;
• Job Scheduling;
• Identity and Access Management.

Cloud Platforms:

• Microsoft Azure;
• Amazon Web Services (AWS).

Infrastructure as Code & Automation:

• Terraform;
• Azure DevOps or GitHub Actions;
• CI/CD Pipelines;
• PowerShell, Python, or Bash scripting.

Security & Networking:

• RBAC;
• Private Endpoints;
• VNET/VPC Design;
• Encryption and Key Management;
• Identity Federation;
• Secrets Management.

Monitoring & Observability:

• Databricks Monitoring;
• Azure Monitor;
• CloudWatch;
• Log Analytics;
• Performance Tuning and Capacity Management.

Preferred Qualifications

• Databricks Certified Platform Administrator;
• Databricks Certified Data Engineer Professional;
• Azure Administrator Associate or Azure Solutions Architect certification;
• AWS Solutions Architect certification;
• Experience implementing enterprise AI and machine learning platforms;
• Experience supporting regulated environments such as Insurance or Financial Services.

Desired Knowledge of AI Features

The ideal candidate should have exposure to one or more of the following:

• Databricks Mosaic AI;
• MLflow;
• Databricks Model Serving;
• Databricks Vector Search;
• Feature Store;
• Retrieval-Augmented Generation (RAG);
• Large Language Models (LLMs);
• AI Governance and Responsible AI Practices;
• AI-enabled Data Engineering Patterns.

Key Competencies

• Cloud Architecture;
• Platform Engineering;
• Infrastructure Automation;
• Operational Excellence;
• Security and Compliance;
• Problem Solving;
• Technical Leadership;
• Communication and Collaboration;
• Continuous Learning and Innovation.

Success Measures

• Stable, secure, and high-performing Databricks platform operations;
• Increased platform automation and reduced operational overhead;
• Improved cloud cost efficiency and resource utilization;
• Successful adoption of enterprise data, analytics, and AI capabilities;
• Strong platform governance, compliance, and reliability;
• High customer satisfaction among data engineering and analytics teams.

Additional information

Modelo de contratação:

• PJ.

Forma de atuação:

• Híbrido(3x por semana presencial no escritório de Pinheiros/SP).