Senior Databricks Engineer — Remote (Secret)

Koniag Government Services; Washington (District of Columbia); Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401(k) retirement plan

Paid time off

Paid parental leave

Life insurance

Disability insurance

Flexible spending accounts

Commuter benefits

Tuition reimbursement

Job summary

Koniag Emerging Technologies, supporting a government client in Washington, DC, seeks a Senior Databricks Engineer to design, develop, and deploy scalable data infrastructure using Databricks and Medallion Architecture. Requires SECRET clearance and strong PySpark/SQL skills; remote position with federal compliance.

You will collaborate across teams, implement CI/CD, and mentor junior staff while building robust data pipelines and dashboards for analytics and BI consumption.

Qualifications

• Active SECRET clearance and ability to pass client suitability checks.
• Proficiency in PySpark and advanced SQL is required.
• 8+ years of data engineering experience with at least 3+ years in a senior Databricks role.
• Deep understanding of Medallion Architecture and modern data warehousing concepts.
• Experience deploying Databricks assets with Azure DevOps (DABs) and YAML pipelines.
• Strong experience with REST API data ingestion and external data integration.
• Cloud experience with Azure Data Lake Storage Gen2 and Azure Key Vault.
• Proficient in Git-based version control and collaborative workflows.
• Excellent written and verbal communication for technical and customer support.

Responsibilities

• Design, develop, and deploy scalable data infrastructure using the Databricks platform and Medallion Architecture.
• Collaborate with project managers, team members, and business owners to gather requirements and ensure data meets customer needs.
• Architect scalable data pipelines with Bronze, Silver, and Gold layers for high quality analytics.
• Establish data quality governance and monitoring dashboards for proactive issue resolution.
• Develop PySpark/SQL ETL/ELT processes; implement DLT and structured streaming pipelines.
• Design and implement RESTful API data ingestion across multiple external systems.
• Automate deployments with CI/CD, using Databricks Asset Bundles and Azure DevOps across Dev/Staging/Prod.
• Create and maintain reusable Python libraries for cross-project use.
• Apply data warehousing concepts to optimize storage and analytics outcomes.
• Monitor and optimize Databricks clusters and Spark jobs for performance and cost efficiency.
• Share data with external BI tools like Power BI and SAP; build dashboards within Databricks when needed.
• Provide mentorship and act as SME for the data engineering team.

Skills

Active SECRET clearance

OOP languages (Java, C#, Python, Java

8+ years Data Engineering; 3+ years in

PySpark & SQL expert

Python libraries packaging (Wheel)

API data ingestion via REST

Databricks CI/CD with Azure DevOps (DA

Azure cloud & ADLS Gen2

Git version control

Written & verbal communication

Job description

Koniag Emerging Technologies, supporting a government client in Washington, DC, seeks a Senior Databricks Engineer to design, develop, and deploy scalable data infrastructure using Databricks and Medallion Architecture. Requires SECRET clearance and strong PySpark/SQL skills; remote position with federal compliance.

You will collaborate across teams, implement CI/CD, and mentor junior staff while building robust data pipelines and dashboards for analytics and BI consumption.