Senior Databricks Data Engineer - Lakehouse & Analytics

Jobtailor; Town of Florida (NY); On-site; USD 150,000 - 190,000; Full time

Job summary

Jobtailor is seeking a senior Data Engineer to own data models, metrics definitions, and algebraic consistency across marketing, revenue, and analytics domains. You will design high-scale Lakehouse architectures, optimize pipelines, and drive governance and observability standards.

The role requires deep Databricks expertise, strong Python/PySpark/SQL skills, and a track record of building production-grade data platforms.

Qualifications

• 7+ years of data engineering, data platform, or analytics engineering experience.
• 5+ years hands-on with Databricks or Spark environments.
• Expertise in Databricks and Lakehouse architecture.
• Strong coding in Python, PySpark, and SQL.
• Experience with Unity Catalog governance and permissions.
• Proven data quality frameworks and observability standards.
• Deep knowledge of Delta Lake internals and distributed processing.
• Experience defining SLAs, anomaly detection, and monitoring.
• Expertise in performance tuning, workload isolation, partitioning, and compute optimization.
• Ability to translate complex architecture into business narratives.
• Familiarity with Terraform, Airflow, and Databricks Workflows.
• Domain experience in Gaming, Hospitality, Loyalty, or Marketing Analytics preferred.

Responsibilities

• Serve as architectural owner for data models, metrics, and semantic consistency.
• Design and maintain curated datasets aligned to business logic and reports.
• Architect and optimize Lakehouse solutions for performance, governance, and cost.
• Define conventions for partitioning, file layouts, and workload orchestration.
• Build reusable ingestion, validation, and orchestration frameworks.
• Establish observability standards including freshness SLAs and anomaly detection.
• Implement automated quality gates and self-healing workflows.
• Own performance tuning across batch, streaming, and analytics workloads.
• Lead incident response and long-term remediation across platforms.
• Mentor engineers and drive cross-functional modernization initiatives.

Skills

Data Engineering

Lakehouse Architecture

Performance Tuning

Python

PySpark

SQL

Data Quality Frameworks

Observability

Distributed Processing

Mentoring

Business Narratives

Tools

Databricks

Unity Catalog

Terraform

Airflow

Databricks Workflows

Delta Lake

Job description

Jobtailor is seeking a senior Data Engineer to own data models, metrics definitions, and algebraic consistency across marketing, revenue, and analytics domains. You will design high-scale Lakehouse architectures, optimize pipelines, and drive governance and observability standards.

The role requires deep Databricks expertise, strong Python/PySpark/SQL skills, and a track record of building production-grade data platforms.