Databricks Architect - Data Pipelines & ML (Remote)

ECS Corporate Services; Fairfax (VA); Hybrid; USD 175,000 - 250,000; Full time

Job summary

ECS Corporate Services seeks a Sr. Databricks Consultant to design, optimize, and fine-tune Databricks environments for advanced analytics, ML, and AI-driven insights.

You will collaborate with clients to deliver measurable improvements in data platforms, pipelines, and governance from our Arlington office or remote setup. The role focuses on Delta Lake, MLflow, and Databricks ML, supporting Professional Services and Managed Cybersecurity Services programs.

Qualifications

• Proven experience working with Databricks in production environments.
• Strong proficiency in Python and/or SQL for data engineering.
• Experience with Apache Spark for distributed data processing.
• Knowledge of data lakehouse architecture and best practices.
• Experience with cloud data platforms (AWS S3, Azure Data Lake, or GCP Storage).
• Hands-on experience building and managing ETL/ELT pipelines.
• Strong understanding of data modeling, data quality, and performance tuning.
• Familiarity with CI/CD pipelines and version control (Git).
• Bachelor's degree; preferably in Computer Science, Information Security, or a related field. Will consider experience in lieu of a degree.
• Must be a US Citizen per contract.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines in Databricks.
• Optimize workflows and clusters for performance and cost efficiency.
• Integrate Databricks with cloud platforms (AWS, Azure, or GCP).
• Implement data governance, security, and compliance standards.
• Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets and insights.
• Troubleshoot issues and drive continuous improvement in data infrastructure.

Skills

Databricks experience

Python/SQL

Apache Spark

Data lakehouse

CI/CD / Git

Data governance/security/compliance

ETL/ELT pipelines

US citizenship

Education

Bachelor's degree

Tools

Delta Lake

MLflow

Git

Cloud platforms (AWS/Azure/GCP)

Apache Spark

Job description

ECS Corporate Services seeks a Sr. Databricks Consultant to design, optimize, and fine-tune Databricks environments for advanced analytics, ML, and AI-driven insights.

You will collaborate with clients to deliver measurable improvements in data platforms, pipelines, and governance from our Arlington office or remote setup. The role focuses on Delta Lake, MLflow, and Databricks ML, supporting Professional Services and Managed Cybersecurity Services programs.