Sr. Azure Data Architect – Databricks

Exavalu; Dallas (TX); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Exavalu, a US-based consulting firm, is seeking a senior Data Architect to design and implement end-to-end data architectures in Azure Databricks, ensuring governance and scalability. The role focuses on cloud-native data ecosystems, hybrid remote work, and collaboration with business stakeholders.

Ideal candidates bring 10+ years in data architecture with hands-on Databricks, Unity Catalog, and ELT tooling experience, delivering high-impact data platforms for Insurance, Healthcare, and

Qualifications

• 10+ years overall experience with data architecture roles.
• 5+ years architecture experience in data engineering.
• Hands-on experience with Azure Databricks and Databricks Unity Catalog.
• Experience with Fivetran or HVR, and DBT for ELT.

Responsibilities

• Design end-to-end data architectures and data ecosystems using Azure-native services and Databricks.
• Build scalable data platforms and robust data frameworks.
• Collaborate with stakeholders to translate business requirements into data solutions.

Skills

Data architecture

Azure Databricks

PySpark

Python

Data governance

RBAC security

Spark SQL

Streaming vs batch

Education

Bachelor's degree in a related field

Tools

Databricks

Fivetran

HVR

DBT

Job description

Dallas, United States

Exavalu is a US based consulting firm specialized in Digital Transformation Advisory and Digital Solution Delivery for select industries like Insurance, Healthcare and Financial Services. Our Headquarters is in California, US with multiple offices across US, Canada, and delivery centres in India. We were founded by Industry executives and Consulting principals with deep industry experience that allows us to bring advisory strength and solution expertise to clients. We’re in a hyper growth trajectory growing at over 100% year on year.

Join our diverse and inclusive team where you will feel valued and motivated to contribute with your unique skills and experience.

Exavalu offers permanent remote working model as we believe in going where the right talent is.

Job Description

Overview :

Location:REMOTE Hybrid

Key Responsibilities

• Design and implement end-to-end data architecture / data ecosystem using Azure-native services and Databricks
  
• Build scalable, high-performance data platforms and robust data frameworks
  
• Partner with stakeholders to translate business requirements into reliable data solutions
  
• Implement data governance principles using Unity Catalog
  
• Create and implement data ingestion and data integration patterns using Medallion Architecture
  
• Develop PySpark notebooks
  
• Create procedures for advanced data transformation
  
• Handle performance tuning in Databricks and Azure SQL DBs
  

Architect end-to-end data solutions supporting:

• Yield improvement
  
• Process control
  
• Predictive maintenance
  
• Operational efficiency
  
• Ensure strict governance, security, scalability, and compliance across the platform
  

Required Experience

• 10+ years of overall experience and 5+ years of architecture experience in data architecture / data engineering roles
  
• Strong hands-on experience with major enterprise data platforms
  

Extensive experience with Azure-native services including:

• Strong hands-on experience with Azure Databricks
  

Required Databricks Expertise

Should have worked with the latest Databricks capabilities including:

Unity Catalog:

• Catalog and schema design
  
• Permissions and security
  
• Managed and external tables
  
• Lineage and governance
  
• CDC implementation
  
• Data transformations
  
• Scaling and performance tuning
  
• Other modern Databricks capabilities and best practices
  

Data Ingestion & ELT Experience

Experience with data ingestion tools such as:

• Fivetran
  
• HVR or equivalent CDC / replication tools
  

Experience with ELT tools such as:

• DBT
  

Familiarity with:

• Batch processing
  

Technical Skills

• Strong expertise in data ingestion, transformation, and orchestration frameworks
  
• Hands-on experience building insurance-specific data models
  
• Proficiency in Python / PySpark and Spark SQL
  
• Strong understanding of streaming vs batch processing
  
• Experience with Spark job performance tuning
  
• Ability to optimize:
  • Data storage formats
    
  • Partitioning strategies
    
  • Schema design
    
• Experience working with:
  • Object stores
    
  • Data lakes
    
  • Cloud compute scaling
    
  • Databricks cluster management
    

Governance, Security & Operations

• Deep understanding of:
  • Data governance
    
  • Security and compliance
    
  • RBAC and attribute-based access control
    
  • Encryption and audit logs
    
  • Data privacy requirements
    
• Experience with operational excellence practices:
  • Monitoring and observability
    
  • Reliability and metrics
    
  • Backup, failover, and disaster recovery strategies
    

Preferred Qualifications

• Prior experience working on Insurance P&C cloud-based data and analytics projects
  
• Strong analytical and problem-solving skills
  
• Experience defining and enforcing best practices for Databricks workloads and Spark-based pipelines
  
• Solid understanding of distributed data processing concepts
  
• Strong communication and stakeholder management skills
  
• Ability to document architecture decisions, trade-offs, and technical recommendations
  

Diversity Inclusion: Exavalu also promotes flexibility, adapting to the needs of employees, customers, and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely. We also offer a welcome back program to help individuals return to the mainstream after a prolonged absence due to health or family reasons. At Exavalu, we are committed to building a diverse and inclusive workforce. We welcome applications for employment from all qualified candidates, regardless of race, colour, gender, national or ethnic origin, age, disability, religion, sexual orientation, gender identity or any other status protected by applicable law. We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career