1150 - Senior Data Platform Architect - Palantir Foundry & Databricks Integration

GoFasti; United States; Teletrabalho; USD 140 000 - 190 000; Tempo integral

Resumo da oferta

GoFasti is seeking a Senior Data Platform Architect to design and implement federated architectures integrating Palantir Foundry and Databricks. You will establish data-sharing standards across Foundry, Databricks, BigQuery, and Cloud Storage, and define patterns using Delta Lake, Iceberg, Unity Catalog, and more.

You will lead modernization of ETL toward metadata-driven cloud-native data products, define governance, lineage, and ownership models, and enable AI workloads across platforms.

Qualificações

• 8+ years in Data Engineering, Architecture or Platform Engineering.
• Extensive Databricks Lakehouse and PySpark expertise.
• Experience with Palantir Foundry platforms and governance.

Responsabilidades

• Design and implement enterprise-scale federated architectures integrating Palantir Foundry and Databricks.
• Establish standards for data sharing across Foundry, Databricks, BigQuery, and Cloud Storage environments.
• Define architecture patterns leveraging Delta Lake, Apache Iceberg, Unity Catalog, Foundry Ontology, Virtual Tables, and Catalog Federation.
• Lead the modernization of legacy ETL architectures toward metadata-driven, cloud-native data products.
• Define enterprise data governance, lineage, metadata, and ownership models across both platforms.
• Create architectural roadmaps supporting AI, machine learning, real-time analytics, and self-service data consumption.
• Design and implement Foundry Databricks Connector solutions, including bulk imports, incremental data synchronization, virtual tables, compute pushdown, and external model integrations.
• Build scalable ingestion, transformation, and enrichment pipelines using PySpark, Spark SQL, Python, Databricks Workflows, Airflow, and Foundry Code Repositories.
• Architect secure connectivity between Palantir Foundry, Databricks, Google Cloud Platform, BigQuery, Cloud Storage, and Enterprise APIs.
• Enable AI and machine learning workloads across Foundry and Databricks environments.
• Serve as the technical authority for enterprise data platform architecture and lead architecture reviews.

Conhecimentos

Data Engineering

Data Architecture

Platform Engineering

Databricks Lakehouse

PySpark

Spark SQL

Python

Data Modeling

Data Governance

Metadata Management

Leadership

Stakeholder Mgmt

Ferramentas

GCP

BigQuery

Cloud Storage

Pub/Sub

IAM

Networking

Descrição da oferta de emprego

We Make Remote Work Remarkable TopTalent from LatAm

Hello! We are GoFasti, a Talent-as-a-Service. GoFasti bridges the gap between world-class developers and designers from LatAm and first-class companies around the globe.

We need an English-fluent Senior Data Platform Architect - Palantir Foundry Databricks Integration, based in Latin America, available to work remotely.

We are looking for someone with exceptional communication and relationship-building skills, who embraces changes while maintaining strong attention to detail. An interested and proactive person, who's constantly learning and improving their skills.

Are you the one we are looking for?

Responsibilities:

• Design and implement enterprise-scale federated architectures integrating Palantir Foundry and Databricks.
• Establish standards for data sharing across Foundry, Databricks, BigQuery, and Cloud Storage environments.
• Define architecture patterns leveraging Delta Lake, Apache Iceberg, Unity Catalog, Foundry Ontology, Virtual Tables, and Catalog Federation.
• Lead the modernization of legacy ETL architectures toward metadata-driven, cloud-native data products.
• Define enterprise data governance, lineage, metadata, and ownership models across both platforms.
• Create architectural roadmaps supporting AI, machine learning, real-time analytics, and self-service data consumption.
• Design and implement Foundry Databricks Connector solutions, including bulk imports, incremental data synchronization, virtual tables, compute pushdown, and external model integrations.
• Build scalable ingestion, transformation, and enrichment pipelines using PySpark, Spark SQL, Python, Databricks Workflows, Airflow, and Foundry Code Repositories.
• Architect secure connectivity between Palantir Foundry, Databricks, Google Cloud Platform, BigQuery, Cloud Storage, and Enterprise APIs.
• Enable AI and machine learning workloads across Foundry and Databricks environments.
• Serve as the technical authority for enterprise data platform architecture and lead architecture reviews.

Requirements:

• 8+ years of experience in Data Engineering, Data Architecture, or Platform Engineering.
• 8+ years of experience with Databricks Lakehouse Architecture.
• Experience implementing or operating Palantir Foundry data platforms.
• Deep expertise in Databricks, PySpark, Spark SQL, Delta Lake, Python, and Data Modeling.
• Strong knowledge of Unity Catalog, Apache Iceberg, Lakehouse Architectures, Data Governance, and Metadata Management.
• Hands-on experience with Google Cloud Platform, including BigQuery, Cloud Storage, Pub/Sub, IAM, and Networking.
• Experience designing highly scalable distributed data systems.
• Strong architecture, leadership, and stakeholder management skills.

It’s a plus:

• Palantir Foundry certification or implementation experience.
• Experience with Ontology development and operational decision platforms.
• Experience with AI/ML infrastructure and model deployment.
• Familiarity with Terrafo