Senior Databricks Solution Architect

Prodapt; Richardson (TX); On-site; USD 180,000 - 260,000; Full time

Job summary

Prodapt seeks a Senior Databricks Solution Architect to lead a Palantir Foundry to Databricks migration, defining a target-state Lakehouse and managing data engineering, AI/GenAI, and analytics capabilities.

You will map existing Palantir pipelines to Databricks services, design scalable data flows, and implement governance, security, and cost-efficient operations across cloud platforms.

Qualifications

• 15+ years of IT experience.
• Extensive Databricks and lakehouse experience.
• Palantir Foundry/AIP migration experience.
• Strong AI/GenAI knowledge including LLMs and embeddings.

Responsibilities

• Define the target-state Databricks architecture for a Lakehouse across Bronze, Silver, Gold.
• Assess Palantir Foundry pipelines and map to Databricks capabilities; identify redesigns.
• Lead migration of data ingestion, transformation, and quality in large-scale environments.
• Architect AI/GenAI solutions including document extraction, embeddings, and model serving.
• Design dashboards and analytics workflows; ensure governance, security, and RBAC compliance.

Skills

Databricks architecture

Lakehouse design

Delta Lake

Apache Spark

Palantir Foundry

Palantir AIP

Unity Catalog

RBAC & security

Cloud platforms (AWS/Azure/GCP)

Data governance

Education

Bachelor's degree in Computer Science or Engineering

Tools

Palantir Foundry

Palantir AIP

Databricks Lakeflow

Unity Catalog

Delta Lake

Apache Spark

Job description

Prodapt is the largest specialized player in the Connectedness industry. As an AI-first strategic technology partner, Prodapt provides consulting, business reengineering, and managed services for the largest telecom and tech enterprises building networks and digital experiences of tomorrow. A ServiceNow-invested company, Prodapt has been recognized by Gartner as a Large, Telecom-Native, Regional IT Service Provider.

Connecting 1.1 billion people and 5.4 billion devices across the globe, Prodapt’s clients include Google, Amazon, PayPal, SoftBank, ServiceNow, Ciena, Verizon, Vodafone, Liberty Global, Liberty Latin America, Claro, Lumen, Windstream, Rogers, Telus, KPN, Virgin Media, British Telecom, Deutsche Telekom, Adtran, Samsung, and many more.

A “Great Place To Work® Certified™” company, Prodapt employs over 6,000 technology and domain experts across the Americas, Europe, India, Africa, & Japan. Prodapt is part of the 130-year-old business conglomerate The Jhaver Group, which employs over 32,000 people across 80+ locations globally.

We are seeking a Senior Databricks Solution Architect to lead the architecture and technical execution of a Palantir Foundry to Databricks migration. The role involves assessing existing Palantir applications, mapping capabilities to Databricks services, defining the target-state Lakehouse architecture, and leading the migration of data engineering, AI/GenAI, analytics, chatbots, and business workflows into scalable, secure, and high-performance solutions.

Key Responsibilities

• Define the target-state Databricks architecture, including scalable Lakehouse design across Bronze, Silver, and Gold layers.
• Assess existing Palantir Foundry pipelines, datasets, applications, workflows, and AI capabilities; develop feature-by-feature mappings to Databricks and identify areas requiring redesign.
• Lead the migration of data ingestion, transformation, data quality, validation, large-scale data comparison, and reconciliation capabilities.
• Design and optimize data pipelines using Databricks Lakeflow, Spark Declarative Pipelines, Apache Spark, and Delta Lake, supporting batch and near-real-time processing.
• Architect AI/GenAI solutions for structured and unstructured document extraction, LLMs, RAG, embeddings, vector search, model serving, and migration of Palantir AIP capabilities.
• Design target-state solutions for dashboards, drill-down reporting, analytical applications, chatbots, conversational analytics, and business workflows.
• Establish enterprise data governance, security, access controls, lineage, and auditing using Unity Catalog and appropriate RBAC policies.
• Optimize Databricks workloads, data pipelines, and compute resources for scalability, performance, reliability, and cost efficiency.
• Lead data engineering and AI/ML teams, conduct architecture reviews, prepare HLD/LLD documents and migration roadmaps, and collaborate with business stakeholders, Palantir SMEs, cloud architects, and application teams.

Requirements

• 15+ years of IT experience.
• Extensive experience in Databricks solution architecture, Lakehouse design, Delta Lake, Apache Spark, and enterprise data engineering.
• Strong expertise in Bronze/Silver/Gold architecture, data ingestion, transformation, data quality, CDC, and batch/near-real-time processing.
• Proven experience with Palantir Foundry and/or Palantir AIP, preferably involving platform migration, capability mapping, and application or workflow redesign.
• Strong knowledge of AI/GenAI solutions, including LLMs, RAG, document intelligence, information extraction, embeddings, vector search, and model serving.
• Hands-on experience with Unity Catalog, enterprise data governance, security, access controls, and cloud platforms such as AWS, Azure, or GCP.
• Experience architecting analytics, dashboards, chatbots, application integrations, and business workflows in enterprise environments.
• Demonstrated technical leadership, architecture documentation, performance optimization, stakeholder management, and ability to lead complex migration initiatives.
• Bachelors degree in Computer Science, Engineering or related.