Senior Data Engineer | Databricks Lakehouse Architect

ONE-Consultants

Breda

On-site

EUR 90,000 - 130,000

Full time

3 days ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

ONE-Consultants is seeking a Senior Data Engineer to design, build, and optimize modern data platforms using the Databricks Lakehouse Platform. The candidate will lead data platform architecture and implement scalable pipelines across Bronze, Silver, and Gold layers with Medallion Architecture principles.

The role requires 10+ years of data engineering experience and deep expertise in Lakehouse, Unity Catalog, and cloud-native solutions.

Qualifications

10+ years of experience in Data Engineering, Data Warehousing, or Big Data platforms.
5+ years of hands-on experience with Databricks in enterprise environments.
Strong expertise in Lakehouse, Medallion Architecture, Unity Catalog.
Experience with AWS, Azure, or GCP.
Programming in Python, SQL, PySpark.
Knowledge of CI/CD and IaC (Terraform).
Experience with data governance and data quality frameworks.
Experience in designing cloud-native data solutions.

Responsibilities

Design and implement scalable, high-performance data platforms using the Databricks Lakehouse architecture.
Lead the development of enterprise data solutions leveraging Bronze, Silver, and Gold layers following Medallion Architecture principles.
Build and maintain batch and streaming data pipelines for large-scale data processing.
Develop reusable data frameworks, accelerators, and engineering best practices.
Optimize data storage, processing, and query performance across cloud environments.
Design, develop, and maintain Databricks workflows using PySpark, SQL, and Python.
Implement and manage Delta Live Tables (DLT) pipelines.
Utilize Unity Catalog for centralized governance, metadata management, access controls, and lineage tracking.
Support advanced analytics, machine learning, and AI workloads on the Databricks platform.
Monitor and troubleshoot production data pipelines and platform performance.
Establish and enforce enterprise data governance standards and policies.
Implement data quality frameworks, validation rules, reconciliation processes, and monitoring solutions.
Ensure compliance with security, privacy, regulatory, and audit requirements.
Drive metadata management, lineage, cataloging, and data stewardship initiatives.
Collaborate with business and governance teams to improve trust and usability of enterprise data assets.
Design and deploy cloud-native data solutions on Azure, AWS, or GCP.
Implement scalable and secure cloud architectures aligned with enterprise standards.
Optimize cloud resource utilization, performance, and cost management.
Integrate cloud-native services with Databricks-based solutions.
Automate deployment, testing, monitoring, and operational processes.
Apply Infrastructure as Code (IaC) practices using tools such as Terraform or similar.
Promote engineering excellence through code reviews, testing strategies, and release management processes.
Provide technical leadership and mentorship to junior and mid-level engineers.
Collaborate with architects, analysts, data scientists, product owners, and business stakeholders.
Contribute to platform roadmaps, architectural decisions, and technology evaluations.
Lead technical discussions and establish best practices across the data engineering organization.