Databricks Archtiect

bfiverecruiting; Juno Beach (FL); On-site; USD 120,000 - 190,000; Full time

Job summary

bfiverecruiting seeks a highly experienced Data Architect with Databricks expertise to define scalable data architectures for enterprise reporting, analytics, AI, and ML initiatives.

The role partners with business stakeholders, data engineers, cloud architects, and leadership to translate requirements into robust roadmaps and implementation plans. Strong focus on security, governance, and cost optimization across cloud data platforms.

Qualifications

• 10+ years in data architecture, data engineering, or enterprise data platforms.
• Hands-on Databricks and cloud-based data architectures.
• Deep lakehouse, Delta Lake, Spark, and distributed processing knowledge.
• Proficient in Python, PySpark, SQL, ETL/ELT, and data integration.
• Experience on AWS, Azure, or Google Cloud.
• Expertise in data modeling, warehousing, and platform design.
• Experience with governance, security, metadata management, and lifecycle management.
• Ability to define standards, evaluate options, and lead design discussions.
• Strong communication to translate requirements into architecture.

Responsibilities

• Define enterprise data architecture and strategies using Databricks and cloud platforms.
• Design scalable lakehouse solutions with Delta Lake and medallion patterns.
• Set standards for ingestion, transformation, modeling, and consumption.
• Design batch and streaming processing with Spark, PySpark, and SQL.
• Define data modeling standards across conceptual, logical, and physical models.
• Establish governance, metadata, lineage, data quality, and access control.
• Implement security using Unity Catalog and cloud-native services.
• Evaluate cloud services and integration patterns; guide engineering teams.
• Create architectural blueprints, reference architectures, and roadmaps.
• Collaborate with architects, security, and data platform teams.

Skills

Data architecture

Databricks

Lakehouse

Python

PySpark

SQL

ETL/ELT

Cloud platforms

Data governance

Communication

Tools

Databricks

Delta Lake

Apache Spark

Job description

We are seeking a highly experienced Data Architect with expertise in Databricks, modern lakehouse architectures, cloud data platforms, and enterprise data strategy. The idealcandidate will define and implement scalable, secure, high-performance data architectures that support enterprise reporting, advanced analytics, AI, and machine learning initiatives.

The Data Architect will collaborate with business stakeholders, data engineering teams, cloud architects, and technology leadership to translate business requirements into robustdata architecture and implementation roadmaps.

Key Responsibilities

• Define enterprise data architecture and technical strategies using Databricks and cloud-native data platforms.
  
• Design scalable lakehouse solutions using Delta Lake, medallion architecture, and appropriate data storage patterns.
  
• Establish architectural standards for data ingestion, transformation, integration, modeling, and consumption.
  
• Design batch and streaming data processing architectures using Apache Spark, PySpark, and SQL.
  
• Define data modeling standards, including conceptual, logical, and physical data models.
  
• Establish data governance, metadata management, lineage, data quality, and access-control frameworks.
  
• Implement governance and security strategies using Unity Catalog and cloud-native identity and security services.
  
• Evaluate and recommend cloud services, data technologies, and integration patterns.
  
• Develop architectural blueprints, reference architectures, technical documentation, and implementation roadmaps.
  
• Guide senior engineers and development teams on architectural decisions and engineering best practices.
  
• Optimize platform performance, scalability, reliability, and overall cloud and Databricks costs.
  
• Collaborate with enterprise architects, security teams, business stakeholders, and data platform teams.
  

Requirements

Required Skills

• 10+ years of experience in data architecture, data engineering, or enterprise data platforms.
  
• Strong hands-on experience with Databricks and cloud-based data architectures.
  
• Deep understanding of lakehouse architecture, Delta Lake, Apache Spark, and distributed data processing.
  
• Strong knowledge of Python, PySpark, SQL, ETL/ELT patterns, and data integration.
  
• Experience designing data platforms on AWS, Microsoft Azure, or Google Cloud.
  
• Expertise in enterprise data modeling, data warehousing, and data platform design.
  
• Experience with data governance, security, metadata management, and data lifecycle management.
  
• Ability to define architecture standards, evaluate technology options, and lead technical design discussions.
  
• Strong communication skills and the ability to translate business requirements into technical architecture.