Senior Databricks Data Engineer - Financial Time-Series

Intone Inc; New York, Northern (NY, KY); Hybrid; USD 150,000 - 190,000; Full time

Job summary

Intone Inc. is seeking a Senior Data Engineer in New York City with deep Databricks expertise to design and own data architecture for complex financial systems.

The role emphasizes end-to-end ownership of data models, ingest, transformation, and publishing layers, and requires leadership in data architecture decisions. You will build full-pipeline solutions, optimize Databricks workflows, and perform production tuning while migrating legacy datasets with parity testing.

Qualifications

• Senior-level ownership of data engineering design and decisions.
• Hands-on production experience with Databricks, Delta Lake, Spark, and orchestration.
• Depth in performance tuning: partitioning, clustering, file sizing, incremental processing.
• Expertise in data modeling and explaining design choices.
• Experience with financial time-series data, as-of joins, bitemporal history, and calendar alignment.
• End-to-end pipeline experience across ingest, transform, and publish layers.
• Experience with large-scale financial systems and data volumes, auditability, and accuracy.
• Familiarity with structured finance datasets (securitizations, ABS/MBS, loan-level and cashflow).
• Strong SQL and Python proficiency; ability to read unfamiliar production code without docs.
• Experience migrating datasets to Databricks with parity testing and cutover.
• Independent work style with limited supervision and ambiguity management.
• Strong communication, collaboration, and technical leadership.

Responsibilities

• Own data model design and architecture decisions for financial instruments and time-series behavior.
• Design and implement full-pipeline data solutions spanning ingest, transform, and publish layers.
• Build and optimize Databricks and Delta Lake workflows using PySpark, Spark SQL, and job orchestration.
• Perform production-level performance tuning across partitioning, clustering, file sizing, and incremental processing.
• Handle financial time-series data correctness concerns including point-in-time accuracy, as-of joins, bitemporal history, late-arriving and restated data, corporate actions, and calendar alignment.
• Lead migration of structured finance datasets from legacy platforms to Databricks, including parity testing and cutover execution.
• Read, reason about, and extract requirements from unfamiliar production code and large legacy systems.
• Collaborate with technical teams and provide leadership on data architecture and implementation decisions.
• Manage work independently with minimal supervision and ambiguous starting requirements.

Skills

Databricks

Delta Lake

Spark (PySpark)

SQL

Python

Data modeling

Time-series data

End-to-end pipelines

Production tuning

Leadership

Tools

Airflow

Databricks Workflows

CI/CD

Unity Catalog

Job description

Intone Inc. is seeking a Senior Data Engineer in New York City with deep Databricks expertise to design and own data architecture for complex financial systems.

The role emphasizes end-to-end ownership of data models, ingest, transformation, and publishing layers, and requires leadership in data architecture decisions. You will build full-pipeline solutions, optimize Databricks workflows, and perform production tuning while migrating legacy datasets with parity testing.