Data Architect

Ventois Inc.; Shrewsbury (MA); On-site; USD 90,000 - 120,000; Full time

Job summary

A technology solutions company is seeking a professional to manage Databricks workspaces using Terraform in Shrewsbury, Massachusetts. The role involves designing, maintaining, and optimizing Databricks environments, developing Terraform modules, and integrating workflows into CI/CD processes. Ideal candidates should have strong skills in infrastructure-as-code, security management, and collaboration with data teams to enable self-service capabilities.

Responsibilities

• Design, provision, and maintain Databricks workspaces using Terraform across multiple environments.
• Develop reusable Terraform modules for Databricks and cloud services.
• Integrate Terraform workflows into CI/CD pipelines for automation.
• Implement Databricks security baselines via Terraform.
• Optimize Databricks platform cost and performance through Terraform configurations.
• Monitor and troubleshoot Databricks infrastructure issues.
• Collaborate with teams to codify Databricks capabilities as Terraform resources.

Job description

Job Responsibilities

• Design, provision, and maintain Databricks workspaces, clusters, jobs, and Unity Catalog using Terraform as infrastructure-as-code across multiple environments.
• Develop and own reusable Terraform modules for Databricks, cloud networking, storage, and security, ensuring consistent, repeatable deployments and minimal configuration drift.
• Integrate Databricks Terraform workflows into CI/CD pipelines (Git-based) to automate environment creation, configuration changes, and promotion across dev, test, and prod.
• Implement and enforce Databricks security baselines (cluster policies, access controls, secrets management) via Terraform, in partnership with cloud and security teams.
• Optimize Databricks platform cost and performance by managing cluster types, pools, autoscaling policies, and job scheduling through Terraform-managed configurations.
• Monitor and troubleshoot Databricks infrastructure issues, building observability and alerting around Terraform-driven changes and platform health.
• Collaborate with data engineering and ML teams to codify new Databricks capabilities (feature stores, workflows, libraries) as Terraform resources, enabling self-service and platform scalability.