Senior Databricks Data Architect & Migration Lead

KPI Partners; United States; On-site; USD 170,000 - 250,000; Full time

Job summary

KPI Partners is seeking a senior technical leader to architect Databricks-based solutions and guide large-scale data platform modernization initiatives. The role blends hands-on engineering with architecture leadership to deliver complex enterprise migrations.

You will define end-to-end data architectures, build scalable pipelines, and establish migration standards from SSIS to Databricks, working with cross-functional teams to ensure quality and alignment with standards.

Qualifications

• Extensive experience designing and implementing enterprise data architectures.
• Strong hands-on experience building complex data pipelines using Databricks.
• Proven experience migrating legacy ETL solutions, particularly SSIS, to Databricks.
• Deep understanding of modern data engineering concepts, including batch and streaming data processing.
• Experience designing reusable data ingestion frameworks and connector architectures.
• Strong expertise in data migration planning, deployment, and cutover strategies.
• Experience reviewing technical designs, architecture documents, and implementation deliverables.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and own the end-to-end target data architecture using Databricks.
• Build scalable, high-performance data pipelines for enterprise data integration.
• Define and establish SSIS-to-Databricks migration standards and best practices.
• Design reusable connector frameworks to support ingestion from multiple data sources.
• Lead migration planning, cutover strategy, and execution for legacy ETL workloads.
• Review architecture, code, and technical deliverables for quality and scalability.
• Serve as the primary technical point of contact for stakeholders and teams.
• Collaborate with engineers and business stakeholders to translate requirements into scalable solutions.
• Optimize performance, reliability, and maintainability of data processing frameworks.
• Drive architecture governance and documentation across the project.

Skills

Databricks

Enterprise data architecture

Data pipelines

ETL migration (SSIS to Databricks)

Batch/Streaming

Data ingestion frameworks

Leadership & stakeholder management

Architecture governance

Tools

Databricks

SSIS

Job description

KPI Partners is seeking a senior technical leader to architect Databricks-based solutions and guide large-scale data platform modernization initiatives. The role blends hands-on engineering with architecture leadership to deliver complex enterprise migrations.

You will define end-to-end data architectures, build scalable pipelines, and establish migration standards from SSIS to Databricks, working with cross-functional teams to ensure quality and alignment with standards.