Databricks Architect (Data Engineering)

Spyro Soft; Deutschland; Remote; EUR 90.000 - 130.000; Vollzeit

Zusammenfassung

Spyro Soft seeks an experienced Databricks Architect to design and lead modern data platforms built on the Databricks ecosystem. You will define architecture standards, establish engineering best practices and support development teams to deliver scalable, secure, high-performing data solutions.

You will design end-to-end Data Lakehouse architectures, lead governance strategies and collaborate with engineers, data scientists and stakeholders to optimize cloud costs and drive CI/CD adoption.

Qualifikationen

• 7+ years of experience in Data Engineering.

Aufgaben

• Design end-to-end Data Lakehouse architectures based on Databricks.
• Define data platform architecture, integration patterns and governance standards.
• Lead architecture decisions around ETL/ELT, batch and streaming data processing.
• Design scalable solutions using Delta Lake, Unity Catalog and Medallion Architecture.
• Collaborate with Data Engineers, Data Scientists, Architects and business stakeholders.
• Define security, data governance and access management strategies.
• Optimize platform performance and cloud costs.
• Support CI/CD implementation and Infrastructure as Code practices.
• Review solution designs and mentor engineering teams.
• Drive adoption of modern Data Engineering best practices.

Kenntnisse

Databricks

Apache Spark

Delta Lake

Medallion Architecture

Unity Catalog

Data Governance

CI/CD

Python

SQL

Cloud Platforms

Orchestration Tools

Data Lakehouse

Stakeholder Management

English Communication

Tools

Databricks

Azure Data Factory

Airflow

Databricks Workflows

dbt

Terraform

Jobbeschreibung

About the role

We are looking for an experienced Databricks Architect to design and lead modern data platforms built on the Databricks ecosystem. In this role you will define architecture standards, establish engineering best practices, and support development teams in delivering scalable, secure and high-performing data solutions.

Responsibilities

• Design end-to-end Data Lakehouse architectures based on Databricks

• Define data platform architecture, integration patterns and governance standards

• Lead architecture decisions around ETL/ELT, batch and streaming data processing

• Design scalable solutions using Delta Lake, Unity Catalog and Medallion Architecture

• Collaborate with Data Engineers, Data Scientists, Architects and business stakeholders

• Define security, data governance and access management strategies

• Optimize platform performance and cloud costs

• Support CI/CD implementation and Infrastructure as Code practices

• Review solution designs and mentor engineering teams

• Drive adoption of modern Data Engineering best practices

Requirements

• 7+ years of experience in Data Engineering

• 3+ years of hands-on experience with Databricks

• Strong expertise in Apache Spark (PySpark and/or Spark SQL)

• Excellent knowledge of Delta Lake and Lakehouse Architecture

• Experience designing enterprise-scale data platforms

• Strong understanding of Medallion Architecture

• Experience with Unity Catalog and data governance

• Experience with cloud platforms (Azure, AWS or GCP)

• Knowledge of orchestration tools (Azure Data Factory, Airflow, Databricks Workflows or similar)

• Experience with CI/CD pipelines and Git-based development

• Strong SQL and Python skills

• Excellent communication and stakeholder management skills

• Fluent English

Nice to have

• Databricks certifications

• Experience with dbt

• Experience with Kafka or other streaming technologies

• Terraform or other IaC tools

• Experience designing AI/ML-ready data platforms