Databricks Architect

Ethereum Technologies; Princeton (NJ); On-site; USD 96,000 - 131,000; Full time

Job summary

Ethereum Technologies in Princeton, NJ seeks a leading Data and AI Architect to design and implement governed, self‑service data solutions on Databricks. You will guide a small engineering team through end‑to‑end architecture, from source data to business answers, combining a federated hub, semantic layer, and conversational AI.

You will define the ontology, metrics, lineage, and governance, publish components to a central hub, and enable secure access via ABAC.

Qualifications

• 10–12 years of experience in data engineering and data architecture.
• At least 3 years of production experience with Databricks, Unity Catalog, Delta Lake, and related tools.
• Experience delivering Genie, text-to-SQL, or LLM agent solutions.
• Strong SQL and Python skills; PySpark experience; Git and CI/CD for data assets.
• Experience with data modelling, conformed dimensions, and cross-team metric reconciliation.

Responsibilities

• Own the end-to-end architecture for conversational self-service on Databricks.
• Design a federated architecture with a central hub and unit-level data context.
• Define semantic and context layer, ontology, keys, rules, lineage, and sources.
• Publish local data definitions to the hub, map to a common model without copying data.
• Establish governance for metric changes, ownership, and versioning.
• Build Unity Catalog metric views, SQL functions for business rules, and ABAC policies.
• Configure Genie spaces, agents, and Genie One with examples and benchmarks.
• Build Agent Bricks that route questions to appropriate agents.
• Use Lakehouse Federation, Delta Sharing, and clean rooms when needed.
• Manage metric views and agent configs in Git with CI/CD via Databricks Asset Bundles.
• Measure answer accuracy with MLflow tracing and evaluation.
• Explain architecture decisions, run workshops, and guide engineers.

Skills

SQL

Python

PySpark

Git

CI/CD

Databricks

Tools

Databricks

Unity Catalog

Delta Lake

Delta Sharing

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Job Title: Databricks Architect

Location: Princeton, NJ

Duration: 12 Months + Extension

Bill Rate: $82/hour (C2C)

Job Type: W-2, 1099, C2C

Client: To Be Discussed Later

Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

About the role:

We’re looking for a hands‑on Lead Data and AI Architect to design and build governed, conversational self‑service data solutions on Databricks. You’ll lead the architecture from source data through to the answers business users receive, combining a federated data platform, a governed semantic layer, and conversational AI. You’ll work with business and technology leaders to establish shared data definitions while allowing individual business units to retain control of their data and context. You’ll also guide a small engineering team and contribute directly to implementation.

What you’ll do:

• Own the end‑to‑end architecture for conversational self‑service on Databricks.
• Design a federated architecture with a central hub for shared definitions and business units or entities that retain their data and context.
• Define the semantic and context layer, including the ontology, common keys, metric views, business rules, entitlements, certification status, lineage, and knowledge sources.
• Design how local definitions and metric views are published to the hub, mapped to a common model, and reused without copying data.
• Establish governance for metric changes and certification, including ownership, versioning, impact analysis, approvals, and effective dates.
• Build Unity Catalog metric views, SQL functions for business rules, and attribute‑based access control (ABAC) policies using row filters and column masks.
• Configure Genie spaces, Genie Agents, and Genie One, including instructions, example SQL, trusted assets, and benchmark questions.
• Build Agent Bricks solutions that route questions to the appropriate agent.
• Use Lakehouse Federation, Delta Sharing, and clean rooms when data needs to remain in place.
• Manage metric views and agent configurations in Git, with CI/CD using Databricks Asset Bundles.
• Measure answer accuracy using MLflow tracing and evaluation.
• Explain architecture decisions to business and technology leaders, run workshops, and guide engineers through delivery.

What you’ll bring:

• 10–12 years of experience in data engineering and data architecture, including at least 3 years as a hands‑on architect on a delivery team.
• At least 3 years of production experience with Databricks, including Unity Catalog, Delta Lake, Databricks SQL, Lakehouse Federation, and Delta Sharing.
• Experience building a governed semantic or metrics layer—such as Databricks metric views or dbt metrics—used by multiple teams.
• Experience delivering a Genie, text‑to‑SQL, or LLM agent solution beyond a demo, including evaluation of answer accuracy.
• Strong SQL and Python skills, PySpark experience, and familiarity with Git and CI/CD for data assets.
• Experience with data modelling, conformed dimensions, common keys across sources, and reconciling metrics against a system of record.
• Experience with access control, lineage, data sharing, and versioning definitions across multiple teams or entities.
• Strong communication skills, with the ability to explain technical decisions clearly, lead workshops, and write effectively.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.