Databricks Data Architect

DynPro Inc.; San Jose (CA); Hybrid; USD 150,000 - 170,000; Full time

Benefits offered by this job

Medical insurance

Vision insurance

Job summary

A technology consulting firm is seeking a skilled Databricks Architect to lead large-scale migrations to the Databricks platform in San Jose, CA. The role requires expert knowledge in data engineering, architecture, and experience with Databricks Lakehouse. The ideal candidate will collaborate with clients and internal teams to define technical blueprints and ensure successful project execution while utilizing a hybrid work model.

Qualifications

• Experience leading data architecture discussions.
• Deep knowledge of Databricks and its components.
• Strong background in building large-scale pipelines.

Responsibilities

• Act as technical lead for Databricks migration projects.
• Define architecture for data ingestion and analytics.
• Collaborate with infrastructure and DevOps teams.

Skills

Data engineering and architecture

Databricks Lakehouse Architecture

ETL/ELT pipelines

AWS stack (S3, Redshift, EMR)

Communication and leadership

Education

8-10+ years of experience

3+ years on Databricks

Tools

Databricks

PySpark

Kafka

Kubernetes

Job description

This range is provided by DynPro Inc.. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$150,000.00/yr - $170,000.00/yr

Direct message the job poster from DynPro Inc.

Tech Recruiting at DynPro - Hiring Exceptional, Innovative and Amazing Talents!

Location: San Jose / Bay Area, CA (Hybrid – Onsite as required)

Employment Type: Full-Time with DynPro

About the Role

We are looking for a highly skilled Databricks Architect to lead the design and technical planning for a large-scale migration to the Databricks platform. This is a strategic pre-execution role where you will drive technical architecture discussions with client stakeholders and internal teams to lay the foundation for a successful project. The role involves working closely with our leadership and the client’s technical leadership to define the migration blueprint, leverage accelerators like Turgon, and prepare the Statement of Work (SOW) for execution.

Responsibilities

• Act as the technical lead and engage with client architects and engineers to understand the current ecosystem and design the target state on Databricks.

• Define the architecture for data ingestion, ETL/ELT pipelines, data lakehouses, and advanced analytics using the Databricks Lakehouse Platform.

• Evaluate existing systems (e.g., AWS Redshift, Spark on EMR, GKE, Kafka) and design migration strategies to Databricks with minimal disruption.=

• Develop architecture artifacts including data flow diagrams, integration patterns, and component-level designs.

• Identify reusable components, best practices, and automation opportunities to accelerate delivery using tools such as Turgon.

• Collaborate with infrastructure, security, and DevOps teams to define cluster sizing, deployment models, CI/CD pipelines, and cost optimization.

• Support SOW and project scoping discussions with effort estimations and risk identification.

• Stay engaged through early build phases, ensuring architectural integrity during handoff to engineering teams.

Must-Have Skills

• 8-10+ years of experience in data engineering and architecture, with at least 3+ years on Databricks.

• Deep knowledge of Databricks Lakehouse Architecture, including Unity Catalog, Delta Lake, DBFS, MLflow, and structured streaming.

• Strong background in building large-scale ETL/ELT pipelines using PySpark, SparkSQL, dbx, and Databricks Workflows.

• Hands-on with AWS stack (S3, Redshift, EMR, Lambda) or Azure equivalents.

• Experience integrating Databricks with Kafka, Airflow, CI/CD pipelines, and container orchestration platforms like Kubernetes.

• Familiar with security frameworks, role-based access control, and compliance in enterprise data environments.

• Strong communication and leadership skills, capable of driving discussions with senior architects and business leaders.

Nice to Have

• Experience using Turgon or other internal accelerators in migration/modernization projects.

• Prior experience in manufacturing, semiconductor, or hardware-driven enterprises.

• Exposure to Computer Vision and Edge AI deployments integrated with Databricks (a plus, not mandatory).

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering, Information Technology, and Strategy/Planning

• Industries

  IT Services and IT Consulting, Consumer Services, and Computer and Network Security

Referrals increase your chances of interviewing at DynPro Inc. by 2x

Inferred from the description for this job

Medical insurance

Vision insurance

Get notified about new Data Architect jobs in San Jose, CA.

San Jose, CA $224,000.00-$410,000.00 1 week ago

Mountain View, CA $149,000.00-$242,000.00 2 weeks ago

Redwood City, CA $180,000.00-$230,000.00 2 weeks ago

Database Product Organization - IC6 AI Architect

Redwood City, CA $116,400.00-$338,500.00 3 days ago

San Jose, CA $181,000.00-$354,200.00 1 week ago

San Jose, CA $155,500.00-$376,000.00 6 days ago

San Francisco Bay Area $160,000.00-$180,000.00 4 weeks ago

Sunnyvale, CA $218,000.00-$306,000.00 1 week ago

Santa Clara, CA $199,000.00-$299,000.00 6 days ago

San Jose, CA $199,800.00-$374,400.00 7 months ago

Sunnyvale, CA $182,000.00-$242,000.00 2 weeks ago

Sunnyvale, CA $188,000.00-$330,000.00 1 week ago

San Jose, CA $150,000.00-$180,000.00 1 month ago

Senior Software Architect - Data Center Systems

Principal Solutions Architect - Silicon Photonics

Mountain View, CA $200,000.00-$260,000.00 1 week ago

Sunnyvale, CA $169,000.00-$338,000.00 1 day ago

Principal Data Center Solutions Architect

San Jose, CA $160,000.00-$200,000.00 15 hours ago

Architect / Technical Lead - System / Packaging / Thermal

Santa Clara, CA $128,670.00-$254,120.00 5 days ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.