GenAI Data Architect Lead | Databricks & ML Pipelines

Karsun Solutions, LLC; Georgia; On-site; USD 160,000 - 175,000; Full time

Job summary

Karsun Solutions, LLC is seeking a Lead Data Architect to design, build and operate enterprise data platforms powering GenAI and AI/ML use cases across cloud environments.

You will own end-to-end data pipelines, model training and deployment pipelines, and governance with Unity Catalog. Strong Databricks, Spark, Python, and AWS experience are essential for delivering scalable, cost-efficient solutions.

Qualifications

• BA/BS in CS or related field required.
• 8+ years hands-on data eng/platform architecture experience.
• 3+ years in architect or lead role.
• Active AWS ML certification or equivalent AWS cert.

Responsibilities

• Architect enterprise data platforms (batch + streaming) for ML/LLM/GenAI work.
• Design Databricks workspaces, Unity Catalog, Delta Lake patterns, and tuning.
• Build end-to-end data pipelines using PySpark, Databricks Jobs, Spark SQL, Delta Lake.
• Implement ML/LLM training, fine-tuning, deployment, and monitoring pipelines (MLOps).
• Develop vectorless/vector embeddings pipelines and retrieval for RAG.”
• Define data schemas, governance, lineage, access controls, APIs using Unity Catalog.
• Optimize storage, compute, and query patterns for cost and performance.
• Collaborate with engineers, data scientists, product owners, security.
• Mentor teams; perform architecture reviews and deep-dives.
• Institute observability, metrics, logging, data quality tests.

Skills

Python

PySpark

Databricks

Spark SQL

SQL

CI/CD

Terraform

IAM

Data governance

ML/LLM pipelines

Education

Bachelor’s degree in CS/CE/IT

Tools

Databricks

Unity Catalog

Delta Lake

Job description

Karsun Solutions, LLC is seeking a Lead Data Architect to design, build and operate enterprise data platforms powering GenAI and AI/ML use cases across cloud environments.

You will own end-to-end data pipelines, model training and deployment pipelines, and governance with Unity Catalog. Strong Databricks, Spark, Python, and AWS experience are essential for delivering scalable, cost-efficient solutions.