Lead Databricks Data & AI Architect

International Business Machines Corporation; Paramus (NJ); Remote; USD 170,000 - 230,000; Full time

Job summary

IBM Corporation is seeking a Databricks Solution Architect to lead the end-to-end data modernization program. You will define target data and AI architecture across AWS/GCP/Azure, drive data migration and modeling, and ensure secure cross-cloud data movement with governance guardrails.

Expect collaboration with executives and stakeholders worldwide. The role requires hands-on expertise with Spark, Delta Lake, MLflow, and Unity Catalog, plus strong experience in GenAI, MLOps, and AI governance.

Qualifications

• Extensive experience designing enterprise-scale data and AI architectures.
• Proven ability to lead cross-cloud data platform programs.
• Hands-on with Spark, Delta Lake, and Databricks.
• Strong knowledge of ML/GenAI, MLOps, and governance.
• Excellent communication with executives.

Responsibilities

• Define the target data and AI architecture across cloud platforms (AWS/GCP/Azure).
• Lead the data migration architecture including patterns, sequencing, cutover and risk management.
• Own data architecture and data modeling standards across lakehouse layers.
• Define data engineering patterns for ingestion, transformation, orchestration, and cost control.
• Architect the reporting and analytics layer with semantic models and governed metrics.
• Design the ML/GenAI lifecycle, including MLOps and observability.
• Architect secure, cross-cloud data movement across hyperscalers with cost/performance balance.
• Ensure use of data products from lakehouse and integration with SAP data sources.
• Define governance guardrails for AI use cases and data privacy.

Education

Databricks Certification

Tools

Databricks

Spark (PySpark/Scala)

Delta Lake

MLflow

Unity Catalog

Kafka

Amazon EventBridge/SQS

Job description

IBM Corporation is seeking a Databricks Solution Architect to lead the end-to-end data modernization program. You will define target data and AI architecture across AWS/GCP/Azure, drive data migration and modeling, and ensure secure cross-cloud data movement with governance guardrails.

Expect collaboration with executives and stakeholders worldwide. The role requires hands-on expertise with Spark, Delta Lake, MLflow, and Unity Catalog, plus strong experience in GenAI, MLOps, and AI governance.