Senior Databricks Platform Engineer - Remote Lakehouse Expert

General Dynamics Information Technology, Inc.; Oregon; Hybrid; USD 187,000 - 252,000; Full time

Benefits offered by this job

Data governance

Remote work

Job summary

General Dynamics Information Technology, Inc. is seeking a Data Platform Engineer to lead design, configuration, and operation of the Databricks Lakehouse for AO's CMM program.

You will shape data governance, pipelines, and analytics across a cloud-based environment to enable reliable decisions. The role requires hands-on expertise with Unity Catalog, RBAC, Terraform, Python, SQL, and Spark tuning, plus collaboration with security and governance teams.

Qualifications

• Extensive experience implementing and administering the Databricks Lakehouse Platform.
• Experience with performance engineering, scalability testing, and tuning of Databricks data platform.
• Strong expertise in SQL, Spark tuning, and workload management.
• Hands-on experience with CI/CD pipelines and IaC automation.

Responsibilities

• Design, configure, and maintain the enterprise Databricks Lakehouse Platform.
• Administer and optimize Databricks compute, clusters, SQL warehouses, serverless capabilities, and workload management.
• Develop and optimize data pipelines, ETL/ELT processes, and ingestion frameworks using Spark, Delta Lake, and Databricks Workflows.
• Create, maintain, and govern Unity Catalog catalogs, schemas, tables, roles/groups, and RBAC.
• Implement and manage Unity Catalog, RBAC, and data governance controls.
• Automate platform provisioning, configuration, and deployments using Terraform, Python, SQL, and Databricks APIs.
• Implement platform monitoring, logging, observability, and cost optimization.
• Perform Spark and SQL performance tuning, scalability testing, and trouble-shooting.
• Collaborate with architecture, security, governance, and data engineering teams; mentor platform users.

Skills

Databricks Lakehouse

SQL performance tuning

Python scripting

Bash scripting

CI/CD pipelines

Docker

Kubernetes

Spark performance tuning

Terraform

RBAC / IAM

Monitoring tools

Azure DevOps

GitHub Actions

Delta Lake

ETL/ELT

Data governance

Security compliance

FedRAMP

Networking

Education

MA/MS or equivalent

Tools

Databricks

Unity Catalog

Delta Live Tables

Databricks Workflows

Grafana

Prometheus

Datadog

CloudWatch

GitHub Actions

Job description

General Dynamics Information Technology, Inc. is seeking a Data Platform Engineer to lead design, configuration, and operation of the Databricks Lakehouse for AO's CMM program.

You will shape data governance, pipelines, and analytics across a cloud-based environment to enable reliable decisions. The role requires hands-on expertise with Unity Catalog, RBAC, Terraform, Python, SQL, and Spark tuning, plus collaboration with security and governance teams.