Senior Databricks Platform Engineer - Remote

ICF International; United States; Remote; USD 99,000 - 203,000; Full time

Job summary

ICF is seeking a Senior Databricks Platform Engineer to design, build, secure, and operate governed Azure Databricks Lakehouse capabilities supporting a cloud-based analytics platform, enterprise AI operations, and emerging data initiatives.

You will develop reliable batch and streaming pipelines, administer Databricks workspaces and access, implement governance, and automate delivery through infrastructure as code and CI/CD.

Qualifications

• BA/BS/MS in Computer Science, Engineering, Data Science or equivalent with 5+ years of data engineering experience.
• Azure Databricks and related Azure data services experience required.
• Expertise in Spark, Delta Lake, Python, SQL and Scala for large-scale ETL/ELT pipelines.
• Experience configuring and managing Databricks workspaces, clusters, and security controls.
• Experience with CI/CD or infrastructure as code for data platforms.

Responsibilities

• Design, configure, and operate Azure Databricks workspaces and governance.
• Build and optimize batch and streaming data pipelines.
• Implement medallion architecture, CDC, and data quality checks.
• Automate Databricks deployments using Terraform, Bundles, and CI/CD.
• Collaborate with architects, security, and mission owners across data initiatives.

Skills

Azure Databricks

Python

SQL

Scala

Apache Spark

Delta Lake

CI/CD

Terraform

GitHub Actions

Azure DevOps

Education

BA/BS/MS in Computer Science, Engineering, Data Science

Tools

Terraform

Databricks Asset Bundles

GitHub Actions

Azure DevOps

Job description

ICF is seeking a Senior Databricks Platform Engineer to design, build, secure, and operate governed Azure Databricks Lakehouse capabilities supporting a cloud-based analytics platform, enterprise AI operations, and emerging data initiatives.

You will develop reliable batch and streaming pipelines, administer Databricks workspaces and access, implement governance, and automate delivery through infrastructure as code and CI/CD.