Lead Data Platform Engineer: Orchestration & Observability

K2 Intelligence, LLC; Northern (KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

K2 Intelligence, LLC seeks a Lead Data Engineer to own data platform patterns and end-to-end data flows. You will design ingestion, orchestration, lineage, data modeling, and data contracts to enable scalable governance and analytics.

In this design-forward role you will set patterns for the team, trace data from source to downstream consumers, and ensure changes propagate cleanly with strong observability.

Qualifications

• 6+ years in data engineering with senior-level ownership of production data platforms.
• Hands-on Databricks (Spark, Delta Lake) and PostgreSQL experience with complex SQL and performance tuning.
• Experience building orchestration for complex, dependency-heavy flows including dependency-aware refresh and rollback.
• Demonstrated data observability and data quality work: automated checks and anomaly detection.
• Experience implementing data lineage and contracts for debugging, impact analysis and audit.
• Strong Python and SQL; solid software engineering fundamentals (VCS, testing, IaC).
• Experience in regulated domains (financial crime, AML, KYC, sanctions) preferred.
• Familiarity with data governance and privacy regimes (ISO, GDPR) preferred.
• Platform design to scale across markets and product lines preferred.

Responsibilities

• Architect orchestration, observability, lineage, and data contracts as reusable platform capabilities.
• Design and evolve logical and physical data models across the platform.
• Build and harden ingestion and transformation pipelines; manage loads into/out of the serving layer.
• Implement automated data-quality checks and anomaly detection.
• Establish data best practices: naming conventions and entity linkage.
• Contribute to governance: PII masking, retention rules, compliance controls by market.
• Mentor engineers and raise the bar on rigor, testing, and operational readiness.

Skills

Python

SQL

Data modeling

Data governance

Regulatory/compliance knowledge

Tools

Databricks

PostgreSQL

Job description

K2 Intelligence, LLC seeks a Lead Data Engineer to own data platform patterns and end-to-end data flows. You will design ingestion, orchestration, lineage, data modeling, and data contracts to enable scalable governance and analytics.

In this design-forward role you will set patterns for the team, trace data from source to downstream consumers, and ensure changes propagate cleanly with strong observability.