Lead Databricks Architect

Sutherland; Dallas (TX); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Databricks expert team

Job summary

Sutherland is seeking an experienced Lead Databricks Architect to drive an enterprise data platform strategy and design a secure, scalable Databricks Lakehouse for the banking sector.

Qualified candidates will have deep Databricks architecture expertise, cloud data platform experience, and a strong background in data governance and regulatory compliance within financial services.

Qualifications

• 10+ years of hands-on Databricks architecture and production experience.
• Strong knowledge of Delta Lake, Unity Catalog, PySpark, Databricks SQL.
• Experience with Azure, AWS, or GCP cloud data platforms.
• Expertise in data governance, RBAC/ABAC, security and regulatory compliance.
• Proficiency in Terraform, CI/CD pipelines, and Databricks Asset Bundles.
• 5+ years of experience in banking or financial services.
• Understanding of core banking, payments, lending, credit risk, AML, fraud detection, and regulatory reporting.

Responsibilities

• Design enterprise-grade Lakehouse architectures using Databricks, Delta Lake, Unity Catalog, and Databricks SQL.
• Build scalable batch and real-time streaming pipelines using PySpark, Structured Streaming, and Auto Loader.
• Lead legacy data platform migrations from Teradata, Oracle, Snowflake, and mainframe environments.
• Implement enterprise data security, access controls, masking, tokenization, encryption, and regulatory compliance.
• Establish CI/CD, DevOps, Infrastructure as Code, and data quality frameworks using Terraform and Databricks Asset Bundles.
• Optimize performance and costs while enabling fraud detection, AML, credit risk, regulatory reporting, and customer analytics.
• Provide technical leadership and collaborate with business, risk, compliance, data engineering, and analytics teams.

Skills

Leadership

Enterprise data architecture

Data governance

Regulatory compliance

Cloud data platforms

Tools

Databricks

Delta Lake

Unity Catalog

PySpark

Terraform

Databricks SQL

Job description

Employment Type: Full-Time

Experience: 10+ years in Databricks | 5+ years in Banking/Financial Services

We are looking for an experienced Lead Databricks Architect to drive enterprise data platform strategy and design a secure, scalable, high-performance Databricks Lakehouse for the banking and financial services industry.

If you have strong expertise in Databricks architecture, cloud data platforms, enterprise data governance, and banking domain solutions, we'd love to connect with you!

Key Responsibilities

• Design enterprise-grade Lakehouse architectures using Databricks, Delta Lake, Unity Catalog, and Databricks SQL.
• Build scalable batch and real-time streaming pipelines using PySpark, Structured Streaming, and Auto Loader.
• Lead legacy data platform migrations from Teradata, Oracle, Snowflake, and mainframe environments.
• Implement enterprise data security, access controls, masking, tokenization, encryption, and regulatory compliance.
• Establish CI/CD, DevOps, Infrastructure as Code, and data quality frameworks using Terraform and Databricks Asset Bundles.
• Optimize performance and costs while enabling fraud detection, AML, credit risk, regulatory reporting, and customer analytics.
• Provide technical leadership and collaborate with business, risk, compliance, data engineering, and analytics teams.

Required Technical Expertise

10+ years of hands-on Databricks architecture and production experience

Strong knowledge of Delta Lake, Unity Catalog, PySpark, Databricks SQL, and Delta Live Tables

Experience with Azure, AWS, or GCP cloud data platforms

Expertise in data governance, RBAC/ABAC, security, and regulatory compliance

Proficiency in Terraform, CI/CD pipelines, and Databricks Asset Bundles

• 5+ years of experience in banking or financial services.
• Understanding of core banking, payments, lending, credit risk, AML, fraud detection, and regulatory reporting.
• Knowledge of financial data governance, auditability, data privacy, and compliance requirements such as SOX, GDPR, and PCI DSS.

Preferred Certifications

• AWS Solutions Architect or Azure Solutions Architect Expert