Senior Databricks Data & AI Solutions Architect

International Business Machines Corporation; Atlanta (GA); A distancia; USD 150.000 - 210.000; Jornada completa

Descripción de la vacante

IBM Consulting seeks a Databricks Solution Architect to define and lead end-to-end architecture for a large data modernization program. You will bring deep expertise across cloud-based engineering, Databricks platform, data migration, data modeling, data architecture, data engineering, and reporting, connecting governed data products to ML and GenAI solutions across hyperscalers.

You will design secure, scalable, and policy-compliant data architectures embedding AI capabilities into enterprise

Formación

• Databricks Certified Data Engineer or Machine Learning Professional.
• Advanced expertise in data/solution architecture, enterprise data migration, data modelling, lakehouse/data-product architecture, reporting, and cloud-based data engineering using Databricks and AWS.
• Hands-on experience with Spark (PySpark/Scala), Delta Lake, MLflow, Unity Catalog, and large-scale batch/streaming workloads.
• Experience with open table formats (Delta Lake, Apache Iceberg) and data-sharing patterns (Delta Sharing, Lakehouse Federation).

Responsabilidades

• Define target data and AI architecture across AWS/Databricks and GCP ML/GenAI services.
• Lead data migration architecture including patterns, sequencing, cutover, and risk management.
• Own data modeling standards covering conceptual, logical, and physical models and lakehouse layers.
• Define data engineering patterns for ingestion, transformation, orchestration, CI/CD, observability, and cost control on cloud platforms.
• Architect reporting and analytics with semantic models and governed metrics.
• Design ML/GenAI lifecycle including feature engineering, model training/serving, and MLOps.
• Architect cross-cloud data movement across AWS, Azure, and GCP with secure patterns.
• Ensure data products from the Databricks lakehouse and SAP data via SAP BTP/BDC Connect.
• Define security, privacy, governance guardrails for AI use cases.
• Provide architectural oversight on initiatives like MySQL to Databricks Lakebase migration.
• Shape roadmap for advanced AI capabilities (agentic AI, frontier models).
• Lead architecture reviews and communicate decisions to execs and tech teams.
• Embed AI services, models, and orchestration with existing systems and data platforms.

Conocimientos

Databricks

Spark PySpark

Delta Lake

MLflow

Unity Catalog

APIs

Event-driven architecture

Security & governance

GenAI experience

Educación

Databricks Certification

Herramientas

Fivetran

Databricks Lakebase

Collibra

Erwin

ER Studio

Descripción del empleo

IBM Consulting seeks a Databricks Solution Architect to define and lead end-to-end architecture for a large data modernization program. You will bring deep expertise across cloud-based engineering, Databricks platform, data migration, data modeling, data architecture, data engineering, and reporting, connecting governed data products to ML and GenAI solutions across hyperscalers.

You will design secure, scalable, and policy-compliant data architectures embedding AI capabilities into enterprise