Senior Data Platform Engineer

Jobtailor; Seattle (WA); On-site; USD 140,000 - 210,000; Full time

Job summary

Jobtailor seeks a data platform engineer to design, build, and enhance Databricks-powered capabilities for quantitative researchers. You will collaborate to evaluate new platform features, improve governance, and automate operational excellence while advancing the target-state architecture.

The role emphasizes AI-assisted coding, reusable data products, and productionizing prototypes for customers, with strong focus on security and observability.

Qualifications

• Hands-on Databricks experience across workspace, notebooks, and jobs and building platform capabilities.
• Strong Python and SQL skills are required.
• Infrastructure as Code with Terraform and modern CI/CD practices.
• Data governance and observability in managed environments (Unity Catalog, lineage, monitoring).
• Advanced workflow orchestration with Apache Airflow; Astro/Astronomer used domestically.
• Experience building reusable data products and self-service data access for researchers.
• AI-first mindset and fluency with agentic coding tools such as Claude Code.
• Full-time availability.

Responsibilities

• Design, build, and enhance platform capabilities within Databricks and related technologies.
• Evaluate and adopt emerging platform features and technologies.
• Run self-assessments and technology evaluations for research initiatives.
• Improve governance, automation, observability, security, and operational excellence.
• Help define target-state architecture for the custom platform.
• Use AI agents to design, build, test, and operate systems with proper verification.
• Build reusable platform capabilities and data products for researchers to access data.
• Accelerate prototypes to production and support Giant Tech customers on Microsoft tech.

Skills

Databricks experience

Python programming

SQL proficiency

Infrastructure as Code

CI/CD

Data governance

Observability

Airflow workflows

AI agentic tools mindset

Full-time availability

Tools

Databricks

Python

SQL

Terraform

CI/CD

Unity Catalog

Astro

Astronomer

Apache Airflow

Claude Code

Job description

• Design, build, and enhance platform capabilities within Databricks and related technologies
• Evaluate and adopt emerging platform features and technologies
• Run self-assessments and technology evaluations on which research initiatives depend
• Improve governance, automation, observability, security, and operational excellence
• Help define the target-state architecture for the custom platform
• Use AI agents and agentic tools to design, build, test, and operate systems, verifying outputs appropriately
• Build reusable platform capabilities and data products for quantitative researchers to discover, access, and consume data
• Accelerate successful prototypes to production
• Support and advise Giant Tech customers on Microsoft and other industry-leading technologies

Requirements

• Hands-on Databricks experience across workspace, notebooks, and jobs, plus building platform capabilities on it (not just using it)
• Strong Python and SQL
• Infrastructure as code with Terraform and modern CI/CD
• Data governance and observability in a managed environment, such as Unity Catalog, lineage, and monitoring
• Strong workflow orchestration experience with Apache Airflow; the company uses Astro / Astronomer
• Experience building and operating data pipelines integrating disparate, heterogeneous data sources
• Experience building reusable, self-service data products for non-engineer end users, including researchers and analysts
• AI-first mindset; fluency with agentic coding tools such as Claude Code
• Full-time availability

Core Competencies

Demonstrates expertise in Databricks platform capabilities, Python, SQL, and Terraform for building and enhancing data products. Proficient in data governance, observability, and workflow orchestration to support quantitative research initiatives.

Highest-signal resume keywords

• Databricks Experience
• Python Programming
• SQL Proficiency
• Infrastructure As Code (Terraform)
• Workflow Orchestration (Apache Airflow)

Hard Skills

• Databricks
• Python
• SQL
• Terraform
• CI/CD
• Data Governance
• Observability
• Apache Airflow
• Data Pipeline Development
• AI Agentic Tools

Industry Keywords

• Data Products
• Quantitative Research
• Operational Excellence
• Automation
• Security

Tools & Technologies

• Unity Catalog
• Astro
• Astronomer