Databricks Engineer - Remote-First AI & Data Pipelines

RevStar Consulting Inc; Town of Florida, Northern (NY, KY); Hybrid; USD 140,000 - 190,000; Full time

Benefits offered by this job

Paid Time Off

Remote-First Working Environment

Comprehensive Health Coverage

401(k) Retirement Plan

Annual Learning & Development Stipend

Peer Mentorship & Coaching

Professional Growth Opportunities

Company Outings & Volunteer Activities

Collaborative, Innovative Culture

Job summary

RevStar Consulting Inc is seeking a hands-on Databricks Engineer to design, build, and optimize data pipelines and AI solutions on a cloud-agnostic platform.

You will work with architects, data scientists, and clients to implement scalable data architectures using Spark, Delta Lake, and MLflow, across ingestion, transformation, and production deployment. The role emphasizes security, governance, and DevOps/MLOps practices.

Qualifications

• 3+ years of hands-on data engineering with cloud-native architectures.
• 2+ years with Databricks, Spark, Delta Lake, and MLflow.
• Databricks Certification (Mandatory).
• Proficiency in Python and SQL; Spark-based frameworks.
• Experience building and optimizing large-scale ETL/ELT pipelines.
• Knowledge of Lakehouse architecture and cloud-agnostic data solutions.
• Familiarity with CI/CD and IaC for Databricks.

Responsibilities

• Develop and optimize data pipelines using Spark and Delta Lake in Databricks.
• Implement ETL/ELT workflows for ingestion, transformation, and storage.
• Design scalable Lakehouse architectures across structured and unstructured data.
• Integrate Databricks with cloud storage for data management.
• Optimize Spark jobs for performance, cost, and latency.
• Implement monitoring, testing, and data quality processes.
• Support ML model training/deployment with MLflow and MLOps practices.
• Ensure data security, governance, and compliance best practices.

Skills

Python

SQL

Spark

Databricks

MLflow

CI/CD

DevOps/MLOps

Education

Databricks Certification (Mandatory)

Tools

Terraform

Databricks CLI

Job description

RevStar Consulting Inc is seeking a hands-on Databricks Engineer to design, build, and optimize data pipelines and AI solutions on a cloud-agnostic platform.

You will work with architects, data scientists, and clients to implement scalable data architectures using Spark, Delta Lake, and MLflow, across ingestion, transformation, and production deployment. The role emphasizes security, governance, and DevOps/MLOps practices.