Data Platform Engineer - Databricks

Changeis; Berlin; Hybrid; EUR 90.000 - 140.000; Vollzeit

Zusammenfassung

Changeis is seeking an experienced Databricks administrator to manage and optimize our enterprise Databricks environment in a remote arrangement. The ideal candidate has six+ years in cloud data platforms, data engineering infrastructure, or platform administration, with hands-on Databricks experience.

You will collaborate with data engineers, data scientists, developers, security and cloud teams to ensure security, reliability, scalability, and cost efficiency across development, testing, and

Qualifikationen

• Bachelor's degree in a technical field.
• Six+ years in cloud data platforms or platform administration.
• Hands-on Databricks administration in an enterprise.
• Experience with Azure Databricks, AWS Databricks, or GCP Databricks.
• Strong understanding of Databricks components and IAM.

Aufgaben

• Administer and support Databricks workspaces, clusters, jobs, notebooks, pools, SQL warehouses, and related services.
• Configure and manage users, groups, roles, permissions, service principals, and access controls.
• Implement security standards using Unity Catalog and RBAC.
• Monitor platform availability, performance, and health.
• Troubleshoot failed jobs, cluster issues, connectivity problems, and performance bottlenecks.
• Configure and optimize clusters, instance pools, autoscaling, and compute policies.
• Support Databricks integrations with cloud storage, databases, BI platforms, APIs, and enterprise apps.
• Manage configurations across development, testing, staging, and production.
• Support CI/CD and Infrastructure as Code deployment processes.
• Develop monitoring dashboards, alerts, audit reports, and operational docs.
• Conduct platform upgrades, configuration changes, testing, and release activities.
• Collaborate with cybersecurity and governance teams for auditing and data protection.

Kenntnisse

Databricks administration

CI/CD

Cloud platforms

Security & IAM

Troubleshooting

Documentation

Stakeholder management

Python scripting

Ausbildung

Bachelor's degree in CS/IT/Engineering/Data Science

Tools

Databricks

Unity Catalog

Terraform

ARM/Bicep/CloudFormation

Azure DevOps

GitHub Actions

Jobbeschreibung

Be part of an exceptional environment where high performers thrive

Job Openings

Location: Remote

Job Id:766

# of Openings:1

Location: Remote

Position Overview

We are seeking an experienced candidate to manage, maintain, secure, and optimize our enterprise Databricks environment. The ideal candidate will have at least six years of experience in cloud data platforms, data engineering infrastructure, or platform administration, including strong hands‑on experience supporting Databricks.

This candidate will work closely with data engineers, data scientists, developers, security teams, and cloud infrastructure teams to ensure the platform is secure, reliable, scalable, and cost‑effective.

Key Responsibilities

• Administer and support Databricks workspaces, clusters, jobs, notebooks, pools, SQL warehouses, and related platform services.
• Configure and manage user accounts, groups, roles, permissions, service principals, and access controls.
• Implement and maintain security standards using Unity Catalog, role‑based access controls, credential management, and data‑governance policies.
• Monitor platform availability, cluster performance, job execution, resource utilization, and system health.
• Troubleshoot failed jobs, cluster issues, connectivity problems, performance bottlenecks, and access‑related incidents.
• Configure and optimize clusters, instance pools, autoscaling, and compute policies to improve performance and control costs.
• Support Databricks integrations with cloud storage, databases, business intelligence platforms, APIs, and enterprise applications.
• Manage platform configurations across development, testing, staging, and production environments.
• Support automated deployment and infrastructure‑management processes using CI/CD and Infrastructure as Code.
• Develop monitoring dashboards, alerts, audit reports, and operational documentation.
• Conduct platform upgrades, configuration changes, testing, and release activities.
• Partner with cybersecurity and governance teams to support auditing, compliance, logging, and data‑protection requirements.
• Establish platform standards, administrative procedures, naming conventions, and operational best practices.
• Provide technical support and guidance to data engineers, analysts, data scientists, and application teams.
• Evaluate platform usage and recommend improvements related to scalability, reliability, performance, and cost optimization.

Required Qualifications

• At least six years of experience in cloud infrastructure, data platform administration, database administration, DevOps, data engineering, or a related technical field.
• Hands‑on experience administering Databricks in an enterprise environment.
• Experience with Microsoft Azure Databricks, AWS Databricks, or Databricks on Google Cloud.
• Strong understanding of Databricks workspaces, clusters, jobs, notebooks, SQL warehouses, instance pools, and compute policies.
• Experience with Unity Catalog, access controls, identity management, service principals, secrets, and data governance.
• Experience monitoring and troubleshooting Databricks jobs, clusters, integrations, and platform performance.
• Working knowledge of Apache Spark and distributed data-processing concepts.
• Experience with cloud storage, networking, security, and identity‑management services.
• Knowledge of SQL and at least one scripting language, such as Python, PowerShell, Bash, or Scala.
• Experience with Git, CI/CD pipelines, and automated deployment processes.
• Experience with Infrastructure as Code tools such as Terraform, ARM templates, Bicep, or CloudFormation.
• Strong troubleshooting, documentation, communication, and stakeholder‑management skills.
• Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.

Preferred Qualifications

• Databricks Certified Data Engineer, Databricks Certified Associate Developer for Apache Spark, or another relevant Databricks certification.
• Cloud certification from Microsoft Azure, Amazon Web Services, or Google Cloud.
• Experience supporting Delta Lake, Delta Live Tables, MLflow, Lakehouse architecture, and Databricks SQL.
• Experience with Azure DevOps, GitHub Actions, Jenkins, or similar CI/CD platforms.
• Experience integrating Databricks with Power BI, Tableau, Snowflake, SQL Server, or other enterprise data platforms.
• Experience supporting regulated, federal, healthcare, financial services, or other highly controlled environments.
• Familiarity with data cataloging, lineage, audit logging, disaster recovery, and business‑continuity planning.

Key Skills

Databricks administration, Unity Catalog, Apache Spark, cloud infrastructure, cluster management, performance optimization, cost management, platform security, identity and access management, CI/CD, Infrastructure as Code, Python, SQL, Terraform, monitoring, troubleshooting, data governance, and technical documentation.