Senior Data Platform Architect

K2 Integrity; New York (NY); On-site; USD 140,000 - 210,000; Full time

Job summary

K2 Integrity is seeking a Lead Data Engineer to own and evolve the data platform, from ingestion to serving layers. You will set patterns for orchestration, observability, lineage, and data contracts to ensure scale and trust across markets and products.

You will mentor engineers, design robust data models, and implement governance controls to meet regulatory requirements while enabling rapid, reliable data flows.

Qualifications

• 6+ years in data engineering with senior ownership of production data platforms.
• Deep hands-on experience with Databricks (Spark, notebooks, Delta Lake, orchestration/workflows) and PostgreSQL (complex SQL, stored procedures, materialized views).
• Proven experience building orchestration for complex dependency-heavy flows, including dependency-aware refresh, error handling, and rollback.
• Demonstrated data observability/ data quality work: automated checks, anomaly and deviation detection, alerting.
• Experience implementing or operating data lineage and understanding its role in debugging, impact analysis, and audit.
• Strong Python and SQL; solid software-engineering fundamentals (version control, testing, IaC).
• Experience in regulated domains (financial crime, AML, KYC) preferred.
• Experience with data contracts and producer/consumer interface design preferred.
• Familiarity with entity resolution and privacy regimes (ISO, GDPR) preferred.
• Experience designing platforms to scale across multiple markets and product lines preferred.

Responsibilities

• Architect orchestration, observability, lineage, and data contracts as reusable platform capabilities, and document patterns for the team.
• Design and evolve logical and physical data models across the platform, balancing consistency with flexibility for new markets.
• Build and harden ingestion and transformation pipelines; manage loads into and out of the operational serving layer.
• Implement automated data-quality and anomaly detection; define what “healthy” looks like for each key dataset.
• Establish data best practices: naming conventions, design standards, entity linkage that other services build on.
• Contribute to governance: PII masking, retention rules, and compliance controls that vary by market.
• Mentor engineers and raise the bar on rigor, testing, and operational readiness.

Skills

Python

SQL

Data modeling

Data governance

Data quality

Data lineage

Tools

Databricks

PostgreSQL

Orchestration tools

Version control

Infrastructure as code

Job description

K2 Integrity is seeking a Lead Data Engineer to own and evolve the data platform, from ingestion to serving layers. You will set patterns for orchestration, observability, lineage, and data contracts to ensure scale and trust across markets and products.

You will mentor engineers, design robust data models, and implement governance controls to meet regulatory requirements while enabling rapid, reliable data flows.