Senior Databricks Data Engineer

Landing Point; Santa Monica (CA); On-site; USD 150,000 - 225,000; Full time

Job summary

Landing Point is seeking a Senior Databricks Data Engineer to build and scale an enterprise Databricks Lakehouse on Azure, delivering trusted data products and robust data pipelines.

You will own platform governance, performance, and cost optimization while mentoring the team and aligning with stakeholders on roadmap and adoption across the firm.

Strong Azure Databricks, Delta Lake, and PySpark skills are required, with 8+ years in data engineering and 3+ years in Databricks.

Qualifications

• 8+ years data engineering experience, including 3+ years with Azure Databricks.
• Strong Python (PySpark) and Spark SQL skills.
• Delta Lake and Databricks Workflows experience.
• Azure data platform fundamentals and data architecture knowledge.
• Leadership, mentoring, and cross-functional collaboration.

Responsibilities

• Build and evolve an Azure Databricks Lakehouse using Delta Lake.
• Develop scalable ingestion and transformation frameworks.
• Create curated, trusted data products with contracts.
• Operationalize pipelines with Databricks Workflows.
• Own performance and cost optimization across workloads.
• Governance and security via Unity Catalog.
• Set engineering standards for the platform.
• Partner with stakeholders on roadmap, adoption, and priorities.
• Provide technical leadership across the team.

Skills

8+ years data engineering

Azure Databricks

Python (PySpark)

Spark SQL

Delta Lake

Databricks Workflows

Azure fundamentals

Mentoring/leadership

Cross-functional communication

Tools

Terraform

Unity Catalog

Databricks

Job description

Company Overview: A leading investment firm with over $185 billion in assets under management is seeking a Senior Databricks Data Engineer to join their team. The role involves building an enterprise-grade data platform on Azure Databricks to support the firm's dynamic business operations.

Job Responsibilities:

• Build and evolve an internal Azure Databricks Lakehouse platform using Delta Lake and standardized architectural patterns.
• Develop scalable ingestion and transformation frameworks for multiple source systems.
• Implement curated, trusted datasets and “data products” with clear contracts.
• Operationalize pipelines using Databricks Workflows (Jobs).
• Own performance and cost optimization across workloads.
• Implement and administer governance and security with Unity Catalog.
• Establish engineering standards for the platform.
• Partner with stakeholders to align platform roadmap, priorities, and adoption.
• Provide technical leadership across the team.

Qualifications:

• 8+ years of data engineering experience, including 3+ years with Azure Databricks.
• Strong proficiency in Python (PySpark) and Spark SQL.
• Strong experience with Delta Lake.
• Experience productionizing workloads with Databricks Workflows (Jobs).
• Strong Azure fundamentals relevant to a data platform.
• Knowledge of data architecture and platform concerns.
• Demonstrated ability to work cross-functionally and communicate technical decisions.
• Demonstrated technical leadership and mentoring experience.
• Databricks Certified Data Engineer Professional certification preferred.
• Experience with CI/CD for Databricks assets and familiarity with Terraform preferred.
• Experience with streaming or near-real-time ingestion and/or Delta Live Tables preferred.
• Familiarity with Databricks SQL and serving data to BI tools is preferred.
• Private equity, credit, or Alternative Asset Management industry experience preferred.

Compensation:

Salary: $150,000 – $225,000, Plus Bonus