Data Architect - Databricks

KPI Partners; United States; On-site; USD 170,000 - 250,000; Full time

Job summary

KPI Partners is seeking a senior technical leader to architect Databricks-based solutions and guide large-scale data platform modernization initiatives. The role blends hands-on engineering with architecture leadership to deliver complex enterprise migrations.

You will define end-to-end data architectures, build scalable pipelines, and establish migration standards from SSIS to Databricks, working with cross-functional teams to ensure quality and alignment with standards.

Qualifications

• Extensive experience designing and implementing enterprise data architectures.
• Strong hands-on experience building complex data pipelines using Databricks.
• Proven experience migrating legacy ETL solutions, particularly SSIS, to Databricks.
• Deep understanding of modern data engineering concepts, including batch and streaming data processing.
• Experience designing reusable data ingestion frameworks and connector architectures.
• Strong expertise in data migration planning, deployment, and cutover strategies.
• Experience reviewing technical designs, architecture documents, and implementation deliverables.
• Excellent communication and stakeholder management skills.

Responsibilities

• Design and own the end-to-end target data architecture using Databricks.
• Build scalable, high-performance data pipelines for enterprise data integration.
• Define and establish SSIS-to-Databricks migration standards and best practices.
• Design reusable connector frameworks to support ingestion from multiple data sources.
• Lead migration planning, cutover strategy, and execution for legacy ETL workloads.
• Review architecture, code, and technical deliverables for quality and scalability.
• Serve as the primary technical point of contact for stakeholders and teams.
• Collaborate with engineers and business stakeholders to translate requirements into scalable solutions.
• Optimize performance, reliability, and maintainability of data processing frameworks.
• Drive architecture governance and documentation across the project.

Skills

Databricks

Enterprise data architecture

Data pipelines

ETL migration (SSIS to Databricks)

Batch/Streaming

Data ingestion frameworks

Leadership & stakeholder management

Architecture governance

Tools

Databricks

SSIS

Job description

KPI Partners is a 5 times Gartner recognized data, analytics, and AI consulting company. We are leaders in data engineering on Azure, AWS, Google, Snowflake, and Databricks. Founded in 2006, KPI has over 400 consultants and has successfully delivered over 1,000 projects to our clients.

About the Role:

We’re looking for a senior technical leader who can architect modern Databricks solutions, establish migration standards, and guide engineering teams through large-scale data platform modernization initiatives. The ideal candidate combines strong hands‑on technical expertise with architecture leadership and experience delivering complex enterprise migration projects.

Key Responsibilities:

• Design and own the end‑to‑end target data architecture using Databricks.
• Build scalable, high‑performance, and reliable data pipelines for enterprise data integration.
• Define and establish SSIS‑to‑Databricks migration standards and best practices.
• Design reusable connector frameworks to support ingestion from multiple data sources.
• Lead migration planning, cutover strategy, and execution for legacy ETL workloads.
• Review architecture, code, and technical deliverables to ensure quality, scalability, and compliance with architectural standards.
• Serve as the primary technical point of contact for stakeholders, engineering teams, and project leadership.
• Collaborate with Data Engineers, Solution Architects, and business stakeholders to translate business requirements into scalable technical solutions.
• Optimize performance, reliability, and maintainability of data processing frameworks.
• Drive architecture governance, technical standards, and documentation across the project.

Required Qualifications:

• Extensive experience designing and implementing enterprise data architectures.
• Strong hands‑on experience building complex data pipelines using Databricks.
• Proven experience migrating legacy ETL solutions, particularly SSIS, to Databricks.
• Deep understanding of modern data engineering concepts, including batch and streaming data processing.
• Experience designing reusable data ingestion frameworks and connector architectures.
• Strong expertise in data migration planning, deployment, and cutover strategies.
• Experience reviewing technical designs, architecture documents, and implementation deliverables.
• Excellent communication and stakeholder management skills.

Why KPI Partners

At KPI, you will be part of a fast‑growing consulting firm at the intersection of data, analytics, AI/ML, and cloud platforms. You will work with leading technology partners, drive impact for global clients, and shape strategic growth in a collaborative, high‑performance environment. KPI is recognized for employee focus, career development, and innovation.