Lead Databricks & AI Architect — Hybrid

Central Business Solutions, Inc; Dallas, New York (TX, NY); On-site; USD 150,000 - 190,000; Full time

Job summary

Central Business Solutions, Inc. is seeking a Senior Databricks & AI Architect/Lead to drive the evolution of a modern data platform and AI roadmap. You will design scalable Spark-based architectures and govern end-to-end data flows.

Initial onsite in Dallas, with potential to hybrid later, leading data engineering teams and shaping AI workflows using Agentic AI, LangChain, and RAG patterns for complex data domains.

Qualifications

• 8+ years in Data Engineering/Architecture with 3+ years in Databricks leadership.
• Expert Spark (PySpark/Scala), Delta Lake, and Databricks SQL.
• Proven AI solution deployment; Agentic AI and LangChain/LlamaIndex.
• Strong data modeling; reusable ETL/ELT templates.

Responsibilities

• Architectural Leadership: design and govern end-to-end data architectures on the Databricks platform.
• AI Innovation: lead design of AI workflows, including multi-agent systems and LLM orchestration.
• Data Engineering Excellence: standardise templates and robust data modelling for structured/unstructured data.
• Governance & Security: ensure designs comply with regulations using Unity Catalog.

Skills

Spark framework

PySpark

Scala

Databricks SQL

Delta Lake

Agentic AI

LangChain

LLM orchestration

Data modeling

ETL/ELT templates

Tools

Unity Catalog

Job description

Central Business Solutions, Inc. is seeking a Senior Databricks & AI Architect/Lead to drive the evolution of a modern data platform and AI roadmap. You will design scalable Spark-based architectures and govern end-to-end data flows.

Initial onsite in Dallas, with potential to hybrid later, leading data engineering teams and shaping AI workflows using Agentic AI, LangChain, and RAG patterns for complex data domains.