Lead Databricks Architect, Data & AI Platform

Ethereum Technologies LLC; Princeton (NJ); On-site; USD 96,000 - 131,000; Full time

Job summary

Ethereum Technologies in Princeton, NJ seeks a leading Data and AI Architect to design and implement governed, self‑service data solutions on Databricks. You will guide a small engineering team through end‑to‑end architecture, from source data to business answers, combining a federated hub, semantic layer, and conversational AI.

You will define the ontology, metrics, lineage, and governance, publish components to a central hub, and enable secure access via ABAC.

Qualifications

• 10–12 years of experience in data engineering and data architecture.
• At least 3 years of production experience with Databricks, Unity Catalog, Delta Lake, and related tools.
• Experience delivering Genie, text-to-SQL, or LLM agent solutions.
• Strong SQL and Python skills; PySpark experience; Git and CI/CD for data assets.
• Experience with data modelling, conformed dimensions, and cross-team metric reconciliation.

Responsibilities

• Own the end-to-end architecture for conversational self-service on Databricks.
• Design a federated architecture with a central hub and unit-level data context.
• Define semantic and context layer, ontology, keys, rules, lineage, and sources.
• Publish local data definitions to the hub, map to a common model without copying data.
• Establish governance for metric changes, ownership, and versioning.
• Build Unity Catalog metric views, SQL functions for business rules, and ABAC policies.
• Configure Genie spaces, agents, and Genie One with examples and benchmarks.
• Build Agent Bricks that route questions to appropriate agents.
• Use Lakehouse Federation, Delta Sharing, and clean rooms when needed.
• Manage metric views and agent configs in Git with CI/CD via Databricks Asset Bundles.
• Measure answer accuracy with MLflow tracing and evaluation.
• Explain architecture decisions, run workshops, and guide engineers.

Skills

SQL

Python

PySpark

Git

CI/CD

Databricks

Tools

Databricks

Unity Catalog

Delta Lake

Delta Sharing

Job description

Ethereum Technologies in Princeton, NJ seeks a leading Data and AI Architect to design and implement governed, self‑service data solutions on Databricks. You will guide a small engineering team through end‑to‑end architecture, from source data to business answers, combining a federated hub, semantic layer, and conversational AI.

You will define the ontology, metrics, lineage, and governance, publish components to a central hub, and enable secure access via ABAC.