Databricks Data Engineer — Secret Clearance & AWS Gov Cloud

NewGen Technologies; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

NewGen Technologies seeks a data engineer to build and operate Databricks pipelines across bronze/silver/gold layers, migrating legacy logic and data. You will gather requirements, monitor jobs, and document workflows to support critical national-security data initiatives.

The role emphasizes batch and streaming ingestion, Delta Lake, and Medallion Architecture within a DoeD-aware AWS Gov Cloud environment, with strong collaboration across teams.

Qualifications

• US Citizenship required.
• Secret Clearance required.
• Excellent interpersonal and prioritization skills.
• Excellent written and oral communication skills.
• Strong data engineering/ETL/ELT or data warehousing background.
• Solid PySpark/Spark SQL and SQL fundamentals.
• Knowledge of Medallion Architecture and Delta Lake.

Responsibilities

• Build and maintain Databricks pipelines (60% of time).
• Gather and analyze requirements (20% of time).
• Monitor and troubleshoot data jobs (10% of time).
• Document processes and workflows (10% of time).

Skills

PySpark

Spark SQL

SQL fundamentals

ETL/ELT development

Data warehousing

Medallion Architecture

Delta Lake

Batch processing

Streaming ingestion

Agile process

Customer service

DoD IT architectures

Tools

Databricks platform

AWS Gov Cloud

Delta Lake

Job description

NewGen Technologies seeks a data engineer to build and operate Databricks pipelines across bronze/silver/gold layers, migrating legacy logic and data. You will gather requirements, monitor jobs, and document workflows to support critical national-security data initiatives.

The role emphasizes batch and streaming ingestion, Delta Lake, and Medallion Architecture within a DoeD-aware AWS Gov Cloud environment, with strong collaboration across teams.