Senior Databricks Consultant: AI & Pipelines (TS/SCI)

ecsfederal; Virginia (MN); On-site; USD 175,000 - 250,000; Full time

Benefits offered by this job

Active TS/SCI-FSP Clearance

US citizen

Job summary

ECS Federal in Arlington, VA is seeking a Sr. Databricks Consultant to design, optimize, and fine-tune Databricks environments for advanced data analytics, machine learning, and AI-driven insights.

You will enable clients to maximize performance, scalability, and value from their data platforms using Delta Lake, MLflow, and Databricks ML. Responsibilities include building scalable ETL/ELT pipelines, optimizing clusters and costs, and ensuring data governance and security.

Qualifications

• Bachelor's degree; computer science, information security, or related field, or equivalent experience.
• US citizenship required; active TS/SCI-FSP clearance preferred.
• Strong experience with Databricks, Python/SQL, and Spark in production environments.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines in Databricks.
• Optimize workflows and clusters for performance and cost efficiency.
• Integrate Databricks with AWS, Azure, or GCP.
• Implement data governance, security, and compliance standards.
• Collaborate with data scientists, analysts, and stakeholders to deliver high-quality datasets and insights.
• Troubleshoot issues and drive continual improvement of data infrastructure.

Skills

Python

SQL

Databricks

Apache Spark

Delta Lake

MLflow

Cloud platforms (AWS/Azure/GCP)

ETL/ELT

Data modeling

Data quality

CI/CD

Git

Education

Bachelor's degree in CS/InfoSec or related field

Tools

Databricks

MLflow

Delta Lake

AWS S3

Azure Data Lake

GCP Storage

Git

Job description

ECS Federal in Arlington, VA is seeking a Sr. Databricks Consultant to design, optimize, and fine-tune Databricks environments for advanced data analytics, machine learning, and AI-driven insights.

You will enable clients to maximize performance, scalability, and value from their data platforms using Delta Lake, MLflow, and Databricks ML. Responsibilities include building scalable ETL/ELT pipelines, optimizing clusters and costs, and ensuring data governance and security.