Remote Federal Databricks Architect

Agility Technologies Inc; Reston (VA); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

401(k) matching

Dental insurance

Health insurance

Paid time off

Parental leave

Vision insurance

Job summary

Agility Technologies Inc. seeks a Databricks Architect to design, implement, and optimize an enterprise data lakehouse platform for federal data solutions.

The candidate will lead data engineering initiatives and act as a technical authority on Databricks, Delta Lake, and Spark. You will collaborate with federal stakeholders, ensure compliance with FedRAMP, NIST, and FISMA, and drive end-to-end data pipelines from ingestion to analytics.

Qualifications

• 5–10 years in data engineering/architecture or distributed data processing.
• Hands-on Databricks (3+ years) experience required.
• Experience with regulated federal data platforms and compliance standards.

Responsibilities

• Design and implement scalable data lakehouse architectures using Databricks for federal data solutions.
• Develop and optimize end-to-end data pipelines, ensuring alignment with federal performance, security, and compliance requirements. Implement data governance and security protocols in accordance with federal standards (e.g., FedRAMP, FISMA, NIST, etc.).
• Collaborate closely with federal stakeholders to gather and analyze requirements and translate them into actionable data solutions. Lead efforts to integrate Databricks with federal cloud services, including AWS GovCloud, Azure Government, or other federal platforms.
• Provide subject matter expertise on Databricks capabilities, including Delta Lake, Databricks SQL, and Apache Spark.
• Optimize Databricks clusters for cost efficiency, high availability, and performance in a secure environment.
• Enable advanced analytics and machine learning workflows for federal use cases, leveraging tools such as MLflow and related frameworks.
• Develop and implement real-time data processing solutions for mission-critical federal applications.
• Stay updated on federal data policies and ensure solutions align with relevant standards and compliance requirements.

Skills

Databricks

Python/Scala/Java

Apache Spark

Delta Lake

FedRAMP security

Cloud platforms

Data governance

Cross-functional leadership

NoSQL

Stakeholder comms

Tools

Terraform

CloudFormation

ARM templates

Job description

Agility Technologies Inc. seeks a Databricks Architect to design, implement, and optimize an enterprise data lakehouse platform for federal data solutions.

The candidate will lead data engineering initiatives and act as a technical authority on Databricks, Delta Lake, and Spark. You will collaborate with federal stakeholders, ensure compliance with FedRAMP, NIST, and FISMA, and drive end-to-end data pipelines from ingestion to analytics.