Freelancer Databricks Machine Learning Engineer

MJV Horizon; United Kingdom; Remote; GBP 90,000 - 130,000; Full time

Job summary

MJV Horizon is seeking an experienced Senior Machine Learning Engineer to architect, build, and scale an enterprise MLOps platform within the Databricks ecosystem.

You will bridge experimental data science and production engineering, leading end-to-end ML pipelines, governance, and CI/CD practices while collaborating with teams across English- and Spanish-speaking markets.

Qualifications

• 5+ years of software/ML engineering experience with at least 3+ years on the Databricks Lakehouse.
• Fluent in English and Spanish to collaborate with regional stakeholders.
• Strong Python skills with advanced SQL.
• Experience with MLflow, scikit-learn, XGBoost, LightGBM, PyTorch.
• Experience structuring environments with Unity Catalog and Delta Lake.
• Experience with CI/CD and IaC using DABs, Terraform.
• Experience with Spark/PySpark and distributed data processing.

Responsibilities

• End-to-End MLOps pipelines: design, build, and scale pipelines using Databricks Workflows, MLflow, and Delta Lake.
• Model serving, low-latency endpoints, and real-time monitoring in production.
• Large-scale data and feature engineering with PySpark and Feature Store.
• Governance, RBAC, lineage, and auditing across ML assets with Unity Catalog.
• Production observability for model performance and data drift.
• CI/CD and IaC modernization using DABs, Terraform, and Git-based pipelines.
• Cross-regional collaboration and mentorship across the Americas in English and Spanish.

Skills

Strong collaboration

Mentorship

Problem solving

Communication

Tools

Databricks Lakehouse

MLflow

Delta Lake

Unity Catalog

Databricks Workflows

Terraform

CI/CD (GitHub Actions/GitLab/GitOps)

Docker

PySpark

Spark

AWS/Azure/GCP

scikit-learn

XGBoost

LightGBM

PyTorch

Job description

About the Role

We are seeking an experienced Senior Machine Learning Engineer with deep hands-on expertise in the Databricks ecosystem to architect, build, and scale our enterprise MLOps platform. In this role, you will bridge the gap between experimental data science and resilient production engineering. You will lead the design and deployment of end-to-end ML pipelines, establish engineering best practices, and collaborate directly with international and cross-functional teams across English- and Spanish-speaking markets.

Key Responsibilities

• End-to-End MLOps Pipelines: Architect and automate robust pipelines for data preparation, model training, evaluation, and automated deployment leveraging Databricks Workflows, MLflow, and Delta Lake.

• Model Serving & Performance Optimization: Deploy, scale, and optimize low-latency real-time endpoints and high-throughput batch pipelines using Databricks Model Serving, monitoring compute resource allocation and cold-start latency.

• Large-Scale Data & Feature Engineering: Design and implement scalable data transformation workflows using PySpark and maintain reusable features using the Databricks Feature Store.

• Governance & Security: Enforce end-to-end data and model lineage, role-based access control (RBAC), and auditing across all ML assets through Unity Catalog.

• Production Observability: Implement comprehensive monitoring systems to track model performance, data drift, concept drift, and system health in real-time.

• CI/CD & Infrastructure as Code: Lead the modernization of deployment pipelines using Databricks Asset Bundles (DABs), Terraform, and CI/CD tools (GitHub Actions, GitLab CI, or Azure DevOps).

• Cross-Regional Collaboration & Mentorship: Partner with regional technical teams, business stakeholders, and clients across the Americas, conducting technical reviews, design discussions, and mentorship in both English and Spanish.

Requirements

Language Proficiency

- Full professional fluency in both English and Spanish (written and verbal) is required to collaborate with distributed regional stakeholders.

Experience

- 5+ years of software/ML engineering experience, with at least 3+ years actively developing on the Databricks Lakehouse platform.

Languages & Core

- Advanced proficiency in Python (OOP, typing, design patterns, testing frameworks) and advanced analytical SQL.

Distributed Computing

- Deep hands-on knowledge of Apache Spark / PySpark, including distributed query optimization, partition tuning, memory management, and debugging skewed datasets.

ML & MLOps

- Strong experience with MLflow (Tracking, Model Registry, Projects), plus standard ML libraries (e.g., scikit-learn, XGBoost, LightGBM, PyTorch).

Enterprise Governance

- Proven hands-on experience structuring environments with Unity Catalog and Delta Lake (ACID transactions, time travel, Z-ordering, Liquid Clustering).

DevOps & Cloud

- Proficiency in automated deployments using Databricks Asset Bundles (DABs) or Terraform, containerization (Docker), and cloud services in AWS, Azure, or GCP.