Business Intelligence jobs in United States

Social network you want to login/join with:

Client:

TechYard

Location:

England, United Kingdom

Job Category:

Other

-

EU work permit required:

Yes

2

23.05.2025

07.07.2025

Techyard is supporting a growing Databricks-partnered consultancy in securing a Databricks Champion Solution Architect to lead the design and delivery of advanced data and AI solutions on the Databricks Lakehouse Platform. This strategic, high-impact role is ideal for a seasoned professional who can operate as both a hands-on architect and a trusted advisor—bridging business vision with technical excellence. You’ll play a pivotal part in driving innovation, embedding best practices, and shaping the consultancy’s data engineering capabilities as they scale their team and client base.

Key Responsibilities:

• Architect and implement end-to-end, scalable data and AI solutions using the Databricks Lakehouse (Delta Lake, Unity Catalog, MLflow).
• Design and lead the development of modular, high-performance data pipelines using Apache Spark and PySpark.
• Champion the adoption of Lakehouse architecture (bronze/silver/gold layers) to ensure scalable, governed data platforms.
• Collaborate with stakeholders, analysts, and data scientists to translate business needs into clean, production-ready datasets.
• Promote CI/CD, DevOps, and data reliability engineering (DRE) best practices across Databricks environments.
• Integrate with cloud-native services and orchestrate workflows using tools such as dbt, Airflow, and Databricks Workflows.
• Drive performance tuning, cost optimisation, and monitoring across data workloads.
• Mentor engineering teams and support architectural decisions as a recognised Databricks expert.
• Demonstrable expertise with Databricks and Apache Spark in production environments.
• Proficiency in PySpark, SQL, and working within one or more cloud platforms (Azure, AWS, or GCP).
• In-depth understanding of Lakehouse concepts, medallion architecture, and modern data warehousing.
• Experience with version control, testing frameworks, and automated deployment pipelines (e.g., GitHub Actions, Azure DevOps).
• Sound knowledge of data governance, security, and compliance, including Unity Catalog.
• Excellent communication, leadership, and problem-solving skills.
• Databricks certifications (e.g., Data Engineer Associate / Professional or Solutions Architect).
• Familiarity with MLflow, dbt, and BI tools such as Power BI or Tableau.
• Exposure to MLOps practices and deploying ML models within Databricks.
• Experience working within Agile and DevOps-driven delivery environments.