Senior Databricks Solutions Engineer

Unison Group; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Unison Group seeks a Senior Consultant – Databricks to design, implement, and optimize data engineering solutions on the Databricks platform. You will build scalable pipelines, ensure data quality, and collaborate with customers and internal teams to deliver robust data lakehouse solutions.

The role requires hands-on experience with Databricks, Apache Spark, Python, and SQL, and a strong ability to translate business requirements into scalable data solutions within a cloud-based Lakehouse

Qualifications

• 6+ years in Information Technology.
• 3+ years hands-on with Databricks.
• Experience in enterprise data engineering solutions.

Responsibilities

• Understand data and analytics requirements with customers and stakeholders.
• Design, develop, and maintain scalable data pipelines with Databricks and Spark.
• Build and optimize ETL/ELT workflows for batch and near real-time processing.
• Implement Data Lakehouse solutions using Bronze, Silver, Gold architecture.
• Develop data transformations with Python, PySpark, and SQL.
• Ensure data quality, reliability, and performance via testing and tuning.
• Troubleshoot data processing and performance issues.
• Collaborate with Solution Architects, Data Engineers, and DevOps to deliver end-to-end solutions.
• Participate in code reviews and follow best practices.
• Create technical docs and assist deployment/production activities.

Skills

Databricks

Python

SQL

Apache Spark

ETL/ELT design

Data Lakehouse

Data modeling

Tools

Azure Data Factory

Airflow

MLflow

Git

CI/CD

Job description

Unison Group seeks a Senior Consultant – Databricks to design, implement, and optimize data engineering solutions on the Databricks platform. You will build scalable pipelines, ensure data quality, and collaborate with customers and internal teams to deliver robust data lakehouse solutions.

The role requires hands-on experience with Databricks, Apache Spark, Python, and SQL, and a strong ability to translate business requirements into scalable data solutions within a cloud-based Lakehouse