Data Bricks Lead

Veridian Tech; Dallas (TX); Hybrid; USD 140,000 - 190,000; Full time

Job summary

Veridian Tech is seeking a highly skilled Databricks Solution Architect to lead the design and implementation of scalable, enterprise-grade data platforms using Databricks in Dallas. This temporary-to-permanent role requires onsite work and collaboration with stakeholders across cloud environments (AWS/Azure/GCP) to deliver end-to-end data solutions.

The ideal candidate combines architectural leadership with hands-on data engineering, governance, and optimization to drive modern data workflows

Qualifications

• Sr. Data Bricks Engineer, AWS, SQL are mandatory skills.
• Experience with Databricks and cloud platforms (AWS/Azure/GCP).
• Strong data engineering and architectural capabilities.

Responsibilities

• Design end-to-end data architectures using Databricks Lakehouse Platform.
• Architect scalable ETL/ELT pipelines, real-time streaming solutions, and analytics.
• Provide governance, security, and cost management across workspaces.

Skills

Databricks

AWS

SQL

Tools

Delta Lake

MLflow

Unity Catalog

Job description

Data Bricks Lead

Dallas, TX.

7+ MONTHS

local to client site; 4 days/wk+

This is a temp to perm role

Skills (EXPERT/ADVANCED/NONE)

• Sr. Data Bricks Engineer
• AWS
• SQL

Questions (must answer YES to ALL)

• Do you have Sr. Data Bricks Engineer experience?
• Will you work onsite at least 4x a week?

Description:

Mandatory Skills: Sr. Data Bricks Engineer, AWS, SQL

Role Summary

We are looking for a highly skilled Databricks Solution Architect to lead the design and implementation of scalable, enterprise-grade data platforms using Databricks. The ideal candidate will combine strong technical expertise in data engineering and cloud platforms (AWS/Azure/GCP) with architectural leadership, solution design capability, and strong stakeholder engagement skills.

Key Responsibilities

1. Solution Architecture & Design
   • Design end-to-end data architectures using Databricks Lakehouse Platform.
   • Architect scalable ETL/ELT pipelines, real-time streaming solutions, and advanced analytics platforms.
   • Define data models, storage strategies, and integration patterns aligned with business and enterprise architecture standards.
   • Provide guidance on cluster configuration, performance optimization, cost management, and workspace governance.
2. Technical Leadership
   • Lead technical discussions and design workshops with engineering teams and business stakeholders.
   • Provide best practices, frameworks, and reusable component designs for consistent delivery.
   • Perform code reviews and provide technical mentoring to data engineers and developers.
3. Stakeholder & Project Engagement
   • Collaborate with product owners, business leaders, and analytics teams to translate business requirements into scalable technical solutions.
   • Create and present solution proposals, architectural diagrams, and implementation strategies.
   • Support pre-sales or discovery phases with technical input when needed.
4. Data Governance, Security & Compliance
   • Define and implement governance standards across Databricks workspaces (data lineage, cataloging, access control, etc.).
   • Ensure compliance with regulatory and organizational security frameworks.
   • Implement best practices for monitoring, auditing, and data quality management.
5. Continuous Improvement & Innovation
   • Stay updated on Databricks features, roadmap, and industry trends.
   • Recommend improvements, optimizations, and modernization opportunities across the data ecosystem.
   • Evaluate integration of complementary technologies (Delta Live Tables, MLflow, Unity Catalog, streaming frameworks, etc.).

Required Skills & Experience

Technical Skills

• Databricks Expertise: Strong hands‑on experience with Databricks (clusters, notebooks, Delta Lake, MLflow, Unity Catalog).
• Cloud Platforms: Experience with at least one cloud provider (AWS, Azure, GCP).
• Data Engineering: Strong proficiency in Spark, Python, SQL, and distributed data processing.
• Architecture: Experience designing large-scale data solutions including ingestion, transformation, storage, and analytics.
• Streaming: Experience with streaming technologies (Structured Streaming, Kafka, Kinesis, EventHub).
• DevOps: CI/CD practices for data pipelines (Azure DevOps, GitHub Actions, Jenkins, etc.).

Soft Skills

• Strong communication skills with the ability to engage both technical and business teams.
• Experience working in Agile environments.
• Ability to simplify complex technical concepts for non-technical audiences.
• Strong analytical, problem‑solving, and decision‑making abilities.

Preferred Qualifications

• Databricks Certified Data Engineer Professional / Architect certification.
• AWS/Azure/GCP cloud architect certifications.
• Experience with BI tools (Tableau, Power BI, Looker).
• Experience in machine learning workflows and ML operations.
• Background in large-scale data modernization or cloud migration projects.

Why Join Us?

• Opportunity to lead high-impact data initiatives using cutting-edge Databricks capabilities.
• Work with a highly skilled team of engineers, architects, and analytics professionals.
• Professional growth opportunities including certifications and advanced architecture training.
• Collaborative environment that values innovation and continuous improvement.