Remote Databricks Architect for Federal Lakehouse

PCI Professional Services LLC; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

PCI Professional Services LLC is seeking a Solutions/Databricks Architect for a remote opportunity. Lead data lakehouse designs using Databricks, build end-to-end data pipelines, and ensure compliance with federal standards. Strong focus on Delta Lake, Spark, and MLflow in a regulated environment.

Ideal candidate has 3+ years with Databricks, experience with GovCloud/Azure Gov, and public trust clearance eligibility. Remote role reporting to the Program Manager.

Qualifications

• Bachelor's degree in a related discipline.
• 5–10 years of experience in data engineering/architecture or distributed data processing.
• 3+ years hands-on experience with Databricks.
• Public Trust (or higher) clearance eligibility.
• Strong knowledge of Databricks capabilities including Delta Lake, Databricks SQL, and Apache Spark.

Responsibilities

• Design and implement scalable data lakehouse architectures using Databricks for federal data solutions.
• Develop and optimize end-to-end data pipelines with federal standards for performance, security, and compliance.
• Collaborate with federal stakeholders to translate requirements into data solutions and integrate Databricks with cloud services (AWS GovCloud, Azure Government).
• Provide SME on Databricks capabilities and optimize clusters for cost, HA, and performance.
• Enable real-time data processing and ML workflows using MLflow and related frameworks.

Skills

Databricks

Delta Lake

Apache Spark

Python/Scala/Java

SQL

NoSQL

MLflow

Education

Bachelor's degree

Tools

Databricks

Terraform

CloudFormation

ARM templates

TensorFlow/PyTorch/Scikit-learn

Job description

PCI Professional Services LLC is seeking a Solutions/Databricks Architect for a remote opportunity. Lead data lakehouse designs using Databricks, build end-to-end data pipelines, and ensure compliance with federal standards. Strong focus on Delta Lake, Spark, and MLflow in a regulated environment.

Ideal candidate has 3+ years with Databricks, experience with GovCloud/Azure Gov, and public trust clearance eligibility. Remote role reporting to the Program Manager.