Senior Databricks Consultant (Remote) - AI & Data Pipelines

ecsfederal; Virginia (MN); Hybrid; USD 175,000 - 250,000; Full time

Job summary

ECS Federal seeks a Sr. Databricks Consultant to join our Arlington, VA team, with option for remote work.

You will design, optimize, and fine-tune Databricks environments for advanced analytics and ML, leveraging Delta Lake, MLflow, and Spark in cloud ecosystems. Responsibilities include building scalable pipelines, optimizing performance and cost, and collaborating with clients to deliver data-driven insights.

Qualifications

• Requires active DHS Public Trust Clearance.
• Proven Databricks experience in production environments.
• Strong Python and/or SQL for data engineering.
• Experience with Apache Spark for distributed processing.
• Familiarity with data lakehouse architectures and best practices.
• Experience with cloud data platforms (AWS S3, Azure Data Lake, or GCP Storage).
• Hands-on experience building and managing ETL/ELT pipelines.
• Understanding of data modeling, data quality, and performance tuning.
• Knowledge of CI/CD pipelines and version control (Git).
• Ability to travel to customer sites as needed.
• Must be a US Citizen per contract.
• Bachelor’s degree or equivalent experience in CS/Info Security or related field.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines in Databricks.
• Optimize Databricks workflows and clusters for performance and cost.
• Integrate Databricks with AWS/Azure/GCP.
• Implement data governance, security, and compliance standards.
• Collaborate with data scientists, analysts, and stakeholders to deliver high-quality datasets.
• Troubleshoot issues and drive continuous improvement in data infrastructure.

Skills

Python

SQL

Apache Spark

Data governance

ETL/ELT pipelines

CI/CD

Travel readiness

Education

Bachelor’s degree in Computer Science or related field

Tools

Databricks

Delta Lake

MLflow

Cloud platforms (AWS/Azure/GCP)

Git

Job description

ECS Federal seeks a Sr. Databricks Consultant to join our Arlington, VA team, with option for remote work.

You will design, optimize, and fine-tune Databricks environments for advanced analytics and ML, leveraging Delta Lake, MLflow, and Spark in cloud ecosystems. Responsibilities include building scalable pipelines, optimizing performance and cost, and collaborating with clients to deliver data-driven insights.