Senior Databricks Consultant

Billennium; Köln; Hybrid; EUR 110.000 - 145.000; Vollzeit

Zusammenfassung

Billennium is seeking a Senior Databricks Consultant to design and build a modern Databricks Lakehouse platform. You will work closely with stakeholders to drive architecture decisions and deliver scalable data solutions in a cloud-based environment.

Responsibilities include designing and optimizing Lakehouse, building data pipelines with Spark, Delta Lake/Unity Catalog, CI/CD, data governance and collaboration with enterprise teams.

Qualifikationen

• Senior hands-on experience with Databricks and Apache Spark.
• Proven experience building Lakehouse platforms.
• Strong PySpark, Python, and SQL skills.
• Experience with Delta Lake, Unity Catalog, and data pipelines.
• Knowledge of cloud architecture, CI/CD, and data governance.

Aufgaben

• Design, build, and optimize a Databricks Lakehouse platform.
• Develop scalable data processing solutions using Apache Spark and PySpark.
• Create and maintain robust data pipelines and ETL/ELT processes.
• Implement and manage Delta Lake and Unity Catalog capabilities.
• Collaborate with business stakeholders and technical teams to gather requirements.
• Support cloud architecture design, data governance, and best practices.
• Contribute to CI/CD implementation and deployment automation.
• Provide consulting expertise and technical guidance throughout the project lifecycle.

Kenntnisse

Databricks

Apache Spark

PySpark

Python

SQL

Delta Lake

Unity Catalog

CI/CD

Data governance

Stakeholder management

Tools

Delta Lake

Unity Catalog

Jobbeschreibung

We are looking for a Senior Databricks Consultant to join a project focused on designing and building a modern Databricks Lakehouse platform. This is an opportunity to work closely with stakeholders, drive architecture decisions, and deliver scalable data solutions in a cloud-based environment.

What You Will Do:

• Design, build, and optimize a Databricks Lakehouse platform
• Develop scalable data processing solutions using Apache Spark and PySpark
• Create and maintain robust data pipelines and ETL/ELT processes
• Implement and manage Delta Lake and Unity Catalog capabilities
• Collaborate with business stakeholders and technical teams to gather requirements
• Support cloud architecture design, data governance, and best practices
• Contribute to CI/CD implementation and deployment automation
• Provide consulting expertise and technical guidance throughout the project lifecycle

Requirements:

Must have:

• Senior hands-on experience with Databricks and Apache Spark
• Proven experience building Lakehouse platforms
• Strong PySpark, Python, and SQL skills
• Experience with Delta Lake, Unity Catalog, and data pipelines
• Knowledge of cloud architecture, CI/CD, and data governance
• Strong consulting and stakeholder management skills

Nice to have:

• Experience with Azure, AWS, or GCP
• Experience working in large enterprise environments
• Databricks or cloud certifications
• Knowledge of data security and compliance standards
• Long-term cooperation with an experienced technology partner
• Challenging international projects and modern technology stack
• Flexible working model and supportive work environment
• Access to knowledge-sharing initiatives and professional development opportunities
• Collaboration with experienced specialists and architects