Remote Senior Databricks Architect for Data Platforms

Koniag Services, Inc.; Washington (District of Columbia); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401K with company matching

Flexible spending accounts

Paid holidays

Three weeks paid time off

Job summary

Koniag Emerging Technologies, LLC, a Koniag Government Services company, seeks a Senior Databricks Engineer with a Secret clearance to support KET and our government customer in Washington, DC. This is a remote position offering extensive benefits and growth.

The role requires 8+ years in data engineering with 3+ years on Databricks, expert PySpark/SQL, and CI/CD automation using Azure DevOps. You will architect data pipelines, ensure data quality, and mentor staff.

Qualifications

• 8+ years of dedicated Data Engineering experience, with 3+ years in Databricks.
• Strong PySpark and SQL expertise.
• Experience automating Databricks deployments with Azure DevOps.
• Proficiency in REST API data ingestion and data warehousing concepts.
• Solid understanding of Medallion Architecture and streaming pipelines.
• Excellent written and verbal communication for customer use.
• Security clearance: Secret required.

Responsibilities

• Collaborate with PMs and stakeholders to gather requirements and ensure data quality.
• Architect and implement scalable data pipelines using Databricks and Medallion Architecture.
• Implement data quality governance and dashboards to monitor issues.
• Develop ETL/ELT processes with PySpark and SQL; work on DLT and structured streaming.
• Design and build robust RESTful data ingestion from external systems.
• Automate deployments using DABs and Azure DevOps across environments.
• Create reusable Python libraries and frameworks for teams.
• Optimize data warehousing and performance of Spark jobs.
• Share data with external BI tools and enable native dashboards in Databricks.
• Mentor team members and establish best practices.

Skills

Databricks

PySpark

SQL

Cloud Azure

CI/CD

Git

Python Libraries

REST APIs

Data Warehousing

Leadership

Tools

Azure DevOps

Databricks Asset Bundles

DLT

Azure Data Lake Storage Gen2

Job description

Koniag Emerging Technologies, LLC, a Koniag Government Services company, seeks a Senior Databricks Engineer with a Secret clearance to support KET and our government customer in Washington, DC. This is a remote position offering extensive benefits and growth.

The role requires 8+ years in data engineering with 3+ years on Databricks, expert PySpark/SQL, and CI/CD automation using Azure DevOps. You will architect data pipelines, ensure data quality, and mentor staff.