Databricks Solutions Architect

Aptonet; Washington (District of Columbia); On-site; USD 140,000 - 190,000; Full time

Job summary

Aptonet is hiring a Databricks Administrator / SME to lead the design, governance, and optimization of the Databricks Lakehouse across the organization. You will serve as the primary technical authority for Databricks, supporting data engineering, analytics, ML, and AI initiatives while driving platform best practices, performance, security, and scalability.

The role requires hands-on Databricks experience, Unity Catalog governance, Spark tuning, and strong Python/SQL skills.

Qualifications

• Bachelor's degree or equivalent experience in a related field.
• 7+ years in Data Engineering, Data Architecture, or related data platform roles.
• 4+ years of hands-on Databricks experience.
• Proven Unity Catalog implementation and cross-workspace governance.
• Deep Spark performance tuning and enterprise-scale Databricks experience.
• Experience building GenAI/LLM solutions with Databricks.
• Proficiency in Python, Scala, and SQL.
• Experience with IaC tools like Terraform.
• Security, networking, and access control expertise for Databricks.
• CI/CD pipelines with modern tooling.

Responsibilities

• Lead design, implementation, and optimization of enterprise Databricks solutions.
• Manage Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, MLflow, and Mosaic AI.
• Develop scalable ingestion, transformation, and orchestration pipelines.
• Optimize Spark workloads, SQL performance, and cluster utilization.
• Enforce governance, security, compliance, and data management standards.
• Support ML/Generative AI initiatives using MLflow, Vector Search, Feature Store, and LLMs.
• Escalate and resolve complex Databricks production issues.
• Integrate Databricks with cloud platforms, orchestration tools, BI, and enterprise apps.
• Implement monitoring, logging, alerts, and dashboards for visibility.
• Collaborate with stakeholders to deliver data and AI solutions, provide technical leadership and training.

Skills

Databricks

Unity Catalog

Spark tuning

Python

SQL

Terraform

CI/CD

Security & governance

Stakeholder management

ML/Gen AI with Databricks

Education

Bachelor's degree

Tools

Terraform

Databricks CLI

GitHub Actions

Azure DevOps

GitLab

Job description

Background Requirements: Must possess or be eligible to successfully complete a Public Trust background investigation.

About the job

We are seeking a Databricks Administrator / Subject Matter Expert to lead the design, governance, optimization, and adoption of the Databricks Lakehouse platform across the organization. This role will serve as the primary technical authority for Databricks, supporting data engineering, analytics, machine learning, and AI initiatives while driving platform best practices, performance optimization, security, and scalability.

Responsibilities

• Lead the design, implementation, and optimization of enterprise Databricks solutions.
• Manage and support key Databricks capabilities including Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, MLflow, and Mosaic AI.
• Design and maintain scalable data ingestion, transformation, and orchestration pipelines.
• Optimize Spark workloads, SQL performance, cluster utilization, and overall platform efficiency.
• Establish and enforce governance, security, compliance, and data management standards.
• Support machine learning and Generative AI initiatives utilizing MLflow, Vector Search, Feature Store, and LLM capabilities.
• Serve as the escalation point for complex Databricks production issues and performance challenges.
• Develop integrations between Databricks, cloud platforms, orchestration tools, BI solutions, and enterprise applications.
• Implement monitoring, logging, alerting, and operational dashboards for platform visibility and reliability.
• Collaborate with Data Engineering, Data Science, Analytics, Infrastructure, and business stakeholders to deliver data and AI solutions.Provide technical leadership, mentoring, training, and platform best practices across the organization.

Required Skills

• Bachelor's degree in Computer Science, Information Technology, Business, or a related field, or equivalent experience.
• 7+ years of experience in Data Engineering, Data Architecture, or related data platform roles.
• 4+ years of dedicated hands-on Databricks experience.
• Proven experience implementing Unity Catalog, including cross-workspace governance and fine-grained security controls.
• Deep expertise with Apache Spark performance tuning and optimization.
• Experience supporting production Databricks environments at enterprise scale.
• Hands-on experience building Generative AI, LLM, or machine learning solutions using Databricks technologies.
• Advanced proficiency in Python, Scala, and SQL.
• Experience deploying and managing Databricks infrastructure using Terraform or other Infrastructure-as-Code tools.
• Experience implementing secure Databricks networking architectures, including Private Link, VPC Peering, encryption, and access controls.
• Strong experience with CI/CD pipelines using GitHub Actions, Azure DevOps, GitLab, Databricks CLI, or Databricks Asset Bundles.
• Strong troubleshooting, architecture, communication, and stakeholder management skills.

Preferred Skills

• Databricks Certified Solutions Architect.
• AWS Certified Solutions Architect or Azure Solutions Architect Expert certification.
• Experience with Mosaic AI, Feature Store, Vector Search, and MLflow.
• Experience designing enterprise-scale data platforms, AI ecosystems, and modern Lakehouse architectures.
• Experience supporting highly regulated environments with strong governance and compliance requirements.