Lead Data Engineer: Spark & Databricks Delivery Lead

Logicbroker, Inc.; New York (NY); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Logicbroker in New York is hiring a Lead Data Engineer to drive delivery across our analytics and data platform initiatives. This hands-on technical lead translates engineering specs into executable work, coordinates contractors, and partners with an Engineering Manager on architectural direction.

You will execute a data warehouse unification plan, design multi-region pipelines, and ensure gold-layer outputs are reliable for downstream analytics engines.

Qualifications

• Strong hands-on experience with Databricks, PySpark and Spark SQL.
• Experience leading delivery for small engineering teams (formal management not required).
• Clear written and verbal communication in a distributed/remote environment.

Responsibilities

• Lead a small delivery team and guide contractor contributions.
• Unblock work and keep scope aligned for steady momentum.
• Refine analytics foundations and gold-layer data models.
• Design and implement Bronze→Silver→Gold Medallion architecture transformations.
• Ensure gold-layer outputs are performant and consumable by analytics engines.
• Translate engineering specs into scoped tickets and sprint plans.
• Review pull requests and maintain quality across deliveries.

Skills

Data modeling

Leadership

Communication

Agile/ Scrum

Tools

Databricks

PySpark

Spark SQL

Job description

Logicbroker in New York is hiring a Lead Data Engineer to drive delivery across our analytics and data platform initiatives. This hands-on technical lead translates engineering specs into executable work, coordinates contractors, and partners with an Engineering Manager on architectural direction.

You will execute a data warehouse unification plan, design multi-region pipelines, and ensure gold-layer outputs are reliable for downstream analytics engines.