Lead Data Engineer: Databricks & Spark Architect

Logicbroker, Inc.; United States; Hybrid; USD 160,000 - 210,000; Full time

Job summary

Logicbroker, Inc. is hiring a Lead Data Engineer to drive delivery across our analytics and data platform initiatives. You will translate engineering specs into executable work, manage contractors, and coordinate with the Engineering Manager.

The role focuses on execution, refining specs into agile workloads, and delivering complex data transformations while maintaining quality and momentum. This position supports a GDPR-first approach and involves collaborating across Databricks environments in

Qualifications

• Hands-on experience with Databricks, PySpark and Spark SQL.
• Strong data modeling skills for analytics and reporting use cases.
• Experience leading delivery for small engineering teams (formal management not required).
• Clear written and verbal communication in a remote environment.

Responsibilities

• Lead a small delivery team and act as a player-coach by owning the most complex data transformations and modeling work, while reviewing and guiding contractor contributions.
• Keep work unblocked, well-scoped, and on track to ensure consistent delivery of momentum.
• Continue building and refining analytics foundations driven by Gold-layer data models.
• Design and implement Medallion architecture transformations (Bronze → Silver → Gold).
• Ensure Gold-layer outputs are well-modeled, performant, and consistently consumable by an external analytics engine.
• Improve the reliability, clarity, and maintainability of existing pipelines.
• Partner with the Engineering Manager to execute an approved Data Warehouse Unification Plan RFC.
• Plan and deliver sprint-based execution, with execution as the primary focus given the RFC is already defined.
• Work across multiple Databricks instances as part of a GDPR-first data strategy.
• Surface execution risks, sequencing concerns, and dependencies early to support successful delivery.
• Translate engineering specifications into clear, scoped tickets, logical sprint plans, and well-sequenced workstreams.
• Assign work appropriately across contractors based on skill level and complexity.
• Implement complex Spark and Databricks transformations directly as needed.
• Review pull requests and uphold quality standards across all delivered work.
• Communicate progress, blockers, and clarity needs clearly and proactively with stakeholders.
• Maintain delivery momentum without unnecessary process overhead.

Skills

Databricks

PySpark

Spark SQL

Data modeling

Delivery lead

Communication

Job description

Logicbroker, Inc. is hiring a Lead Data Engineer to drive delivery across our analytics and data platform initiatives. You will translate engineering specs into executable work, manage contractors, and coordinate with the Engineering Manager.

The role focuses on execution, refining specs into agile workloads, and delivering complex data transformations while maintaining quality and momentum. This position supports a GDPR-first approach and involves collaborating across Databricks environments in