Databricks Solution Architect

EXL

San Francisco (CA)

Hybrid

USD 180,000 - 240,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

EXL is seeking a Databricks Solution Architect to own end-to-end design of a modern data platform on Databricks. Turn business needs into scalable, secure data architectures integrated with existing systems and lead architectural decisions across teams.

You will work with Principal Engineer on translating architecture into detailed designs and pipelines, guiding migration from Hadoop/Hive/EMR, and ensuring cost and security controls are embedded in the platform.

Qualifications

  • 12+ years in data engineering or architecture.
  • 5+ years on Databricks lakehouse.
  • Strong Spark, Delta Lake and SQL.
  • AWS networking, IAM, encryption and multi-account design.
  • Migration planning from legacy Hadoop/Hive/warehouse systems.
  • Ability to choose between batch and streaming (Kafka, Structured Streaming).
  • Disaster recovery and high-availability design.
  • FinOps and Databricks cost and capacity modeling, business case.

Responsibilities

  • Define the enterprise data architecture on Databricks: ingestion, medallion layering, serving and ML, and connections to source systems and legacy warehouse.
  • Run discovery with business and technology stakeholders and turn requirements into solution designs and roadmaps.
  • Design for identity resolution, segmentation, campaign and data sharing with marketing platforms.
  • Design for near-real-time delinquency data, agency and vendor file exchange, and regulatory compliance.

Skills

Databricks Lakehouse
Data Architecture
Spark
Delta Lake
SQL
AWS
Security Design
Migration Planning
Cost Modeling
Governance
Stakeholder Mgmt
Vendor Mgmt
Leadership

Tools

Databricks

Job description

Our passion at EXL is enabling organizations to achieve a measurable, competitive advantage through Data Management. We partner with our clients to design and implement scalable, accessible, and innovative solutions. Our goal is to make sense of data to drive client’s business forward through data strategy, data governance and building cloud data platforms. EXL is a leading data analytics and digital operations and solutions company that partners with clients to improve business outcomes and unlock growth. By bringing together deep domain expertise with robust data, powerful analytics, cloud, artificial intelligence ("AI") and machine learning ("ML"), we create agile, scalable solutions and execute complex operations for the world’s leading corporations in industries including insurance, healthcare, banking and financial services, media, and retail, among others. Focused on driving faster decision-making and transforming operating models, EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect. Headquarters in New York, our team is over 73,000+ strong, with more than 50 offices spanning six continents.

Role Name: Databricks Solution Architect
Employment Type: Full-Time
Mode – Hybrid (2-3 days from Office)
Job Summary
Job Summary
About the Role

We are building our modern data platform on Databricks and need a Solution Architect to own the end-to-end design. You will turn business needs from Risk, Fraud, Finance, Marketing, Collections and Product into scalable, secure data architectures that integrate cleanly with existing systems. You will own architectural decisions across teams and work closely with the Principal Engineer, who turns the architecture into detailed designs and working pipelines.

Key Responsibilities
  • Target-state architecture: Define the enterprise data architecture on Databricks: ingestion, medallion layering, serving and ML, and how they connect to source systems, the legacy warehouse, loan servicing platforms and vendor feeds.
  • Discovery and roadmaps: Run discovery with business and technology stakeholders and turn requirements into solution designs, reference architectures and delivery roadmaps.
  • Marketing architecture: Design for identity resolution, segmentation, campaign and attribution measurement, and compliant data sharing with marketing platforms and vendors.
  • Collections architecture: Design for near-real-time delinquency and contact data, agency and vendor file exchange, and compliant handling of contact preferences and regulatory restrictions.
  • Cross-cutting decisions: Decide on batch vs. streaming, build vs. buy, workspace topology, network and multi-account design, and disaster recovery.
  • Security architecture: Design the security and compliance architecture with InfoSec: Unity Catalog access model, PII/NPI protection, encryption, audit logging and lineage.
  • Migration planning: Plan and sequence the migration of legacy Hadoop/Hive/EMR and warehouse workloads, including cutover strategy and risk mitigation.
  • Cost and business case: Build the cost and capacity model (DBU forecasting, FinOps guardrails) and the business case for platform investments.
  • Governance: Chair the architecture review board and maintain standards, reference patterns and decision records.
  • Vendors: Manage the relationship with the Databricks account team and cloud provider, and lead vendor evaluations and proofs of concept.
Must-Have Skills (candidate must meet all)
  • Experience: 12+ years in data engineering or architecture, with 5+ years on Databricks lakehouse platform.
  • Data platform fundamentals: Strong Spark, Delta Lake and SQL, plus data modeling (medallion, dimensional).
  • AWS architecture: Networking, IAM, encryption and multi-account design on AWS.
  • Secure design: Platform design for PII/NPI, including Unity Catalog access models and lineage.
  • Migration: Migration planning from legacy Hadoop/Hive/warehouse systems.
  • Streaming and batch design: Ability to choose between batch and streaming (Kafka, Structured Streaming) and design for both.
  • Resilience: Disaster recovery and high-availability design.
  • Cost modeling: FinOps and Databricks cost and capacity modeling, including a business case.
  • Communication: Stakeholder management, vendor management and clear architecture documentation.
Good-to-Have Skills (preferred, not required)
  • Banking, lending or other regulated financial services experience
  • Collections data architecture (delinquency, agency and dialer feeds)
  • ML/MLOps architecture on Databricks (feature store, MLflow)
  • Databricks and AWS Solution Architect certifications
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Solution Architect
Databricks Solution Architect

EXL Talent Acquisition Team • San Francisco (CA)

On-site
USD 190,000 - 260,000
Databricks Solution Architect
Databricks Solution Architect

Centraprise • San Francisco (CA)

On-site
USD 180,000 - 260,000
Databricks Data Architect/Lead
Databricks Data Architect/Lead

Celebal Technologies • United States

On-site
USD 180,000 - 240,000
Senior Resident Solution Architect
Senior Resident Solution Architect

Unison Group • San Francisco (CA)

On-site
USD 180,000 - 240,000
Databricks Solution Architect
Databricks Solution Architect

Valiant Technologies LLC • San Francisco (CA)

On-site
USD 83,000 - 118,000
Databricks Solution Architect
Databricks Solution Architect

Quantum Technologies LLC • San Francisco (CA)

On-site
USD 82,000 - 117,000
Databricks Solution Architect
Databricks Solution Architect

Ethereum Technologies LLC • San Francisco (CA)

On-site
USD 82,000 - 117,000
Databricks Solution Architect
Databricks Solution Architect

Virtue America • Florida

On-site
USD 150,000 - 210,000
Databricks Data Architect
Databricks Data Architect

Winfo Solutions • Atlanta (GA)

On-site
USD 140,000 - 210,000
Databricks Solutions Architect
Databricks Solutions Architect

Compunnel Inc. • Florida

On-site
USD 120,000 - 160,000