Senior Databricks Data Engineer - Lakehouse & Streaming

Machine Learning Technologies, LLC; Columbus (OH); On-site; USD 120,000 - 170,000; Full time

Job summary

Machine Learning Technologies, LLC is seeking aSenior Databricks Developer in Columbus, OH to design and implement large-scale data platforms using Databricks and AWS. The role focuses on building production-grade ETL/ELT pipelines, Delta Lake solutions, and real-time data processing.

You will collaborate with data engineering, architecture, and ML teams, applying Data Mesh principles to enterprise platforms and driving cost-effective, high-performance data workflows.

Qualifications

• Experience designing large-scale data platforms on AWS using Databricks.
• Proven ability to build production-grade ETL/ELT pipelines.
• Strong background in Delta Lake and Lakehouse architectures.

Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Databricks and AWS.
• Implement and optimize Delta Lake and Lakehouse architecture.
• Develop high-performance ETL/ELT pipelines using PySpark and Scala Spark.
• Build and manage high-volume, production-grade data processing workflows.
• Implement real-time and streaming data pipelines using Databricks and Delta Live Tables (DLT).
• Implement and support MLOps solutions within the Databricks ecosystem.
• Apply Data Mesh architecture principles to enterprise data platforms.
• Optimize data processing workloads for performance, scalability, and cost efficiency.
• Monitor and troubleshoot production data pipelines to ensure reliability and data quality.
• Collaborate with data engineering, architecture, and ML teams to deliver enterprise data solutions.

Skills

Databricks

AWS ecosystem

Delta Lake

Lakehouse architecture

PySpark

Scala Spark

Streaming pipelines

Delta Live Tables (DLT)

MLOps

Data Mesh

Cost optimization

Performance engineering

Databricks + AWS Data Engineer

Tools

Delta Lake

Delta Live Tables (DLT)

Lakeflow

Lakebase

Job description

Machine Learning Technologies, LLC is seeking aSenior Databricks Developer in Columbus, OH to design and implement large-scale data platforms using Databricks and AWS. The role focuses on building production-grade ETL/ELT pipelines, Delta Lake solutions, and real-time data processing.

You will collaborate with data engineering, architecture, and ML teams, applying Data Mesh principles to enterprise platforms and driving cost-effective, high-performance data workflows.