Databricks Platform Engineer: AWS, Terraform

First Orion; North Little Rock (AR); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Flexible work arrangements

Health and dental/vision options

Generous PTO

Job summary

First Orion is seeking a Data Platform Engineer to implement and operate a robust Databricks platform on AWS. You will build Terraform modules, manage Unity Catalog governance, and develop data pipelines using Python, SQL, and Spark.

You will partner with the Data Platform Architect to validate designs, ensure production readiness, and optimize security, cost, and performance across environments. This role emphasizes collaboration and reliable, scalable data solutions.

Qualifications

• Requires 5+ years in data or platform engineering.
• 3+ years with Databricks and Apache Spark in production environments.
• Strong Terraform experience with modules/state management.
• AWS experience including IAM, S3, networking, monitoring, and security.
• Experience with Databricks Asset Bundles, Git-based development, automated testing, and deployments.
• Strong Python and SQL skills; working knowledge of PySpark and Delta Lake.
• Experience with identity/access management, service principals, groups, least privilege, and SSO/SCIM.
• Ability to translate architecture/governance requirements into maintainable code and procedures.
• Strong communication and collaboration across architecture, engineering, security, and business teams.

Responsibilities

• Implement, operate, and improve the Databricks platform on AWS.
• Build and maintain Terraform modules for Databricks, AWS infra, Unity Catalog, permissions, and integrations.
• Configure and support workspaces, compute policies, SQL warehouses, workflows, jobs, service principals, and platform permissions.
• Implement Unity Catalog governance (catalogs, schemas, storage credentials, external locations, ownership, group-based access).
• Develop pipelines using Python, SQL, Spark, Delta Lake, and Databricks workflows.
• Create lightweight internal apps and governed user-input solutions using approved platform services.
• Monitor platform health, reliability, performance, security, and cost; troubleshoot incidents.

Skills

Communication

Collaboration

Problem solving

Architecture translation

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Terraform

Unity Catalog

AWS

Python

SQL

PySpark

Delta Lake

Apache Spark

Git

CI/CD

Job description

First Orion is seeking a Data Platform Engineer to implement and operate a robust Databricks platform on AWS. You will build Terraform modules, manage Unity Catalog governance, and develop data pipelines using Python, SQL, and Spark.

You will partner with the Data Platform Architect to validate designs, ensure production readiness, and optimize security, cost, and performance across environments. This role emphasizes collaboration and reliable, scalable data solutions.