On-Site DoD Data Engineer: Pipelines, APIs & Databricks

D&G Support Services; St. Louis (MO); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Benefits

401(k)

Bonus and profit sharing

Flexible hours

Education reimbursement

PTO

Job summary

D&G Solutions seeks a Data Scientist/Engineer to design and maintain data pipelines, APIs, and integration solutions for the Joint Petroleum Enterprise. You will enable bulk fuel data management and support data-driven decisions across the DoD enterprise.

Responsibilities include ingest pipelines, COP data integration, data governance, and API development. Strong Spark/Kafka, Databricks, Palantir Ontology experience and DoD familiarity are preferred.

Qualifications

• Bachelor's degree in CS, data engineering, or related field.
• 5+ years in data engineering, software development, or similar.
• Strong SQL and Python skills with experience in data integration.

Responsibilities

• Develop and maintain data pipelines ingesting data into Databricks with enterprise integration.
• Collaborate on COP development and cross-system data integration efforts.
• Review DoD data sources to identify data enrichment opportunities.
• Design scalable ingestion transforming data for APIs and consumption.
• Develop, publish, and maintain APIs for enterprise data access.
• Maintain data dictionaries, metadata, and governance artifacts.
• Manage data structures and engineering processes following best practices.
• Support metadata, modeling decisions, and tool selection in working groups.
• Prepare technical materials for governance forums and leadership.
• Support data modernization initiatives and project delivery.

Skills

SQL

Python

APIs development

Data pipelines

Spark/Kafka

Cloud platforms

Communication

Education

Bachelor's degree in CS or related

Tools

Databricks

Palantir Ontology

Kafka

REST APIs

Maven Smart Systems

Job description

D&G Solutions seeks a Data Scientist/Engineer to design and maintain data pipelines, APIs, and integration solutions for the Joint Petroleum Enterprise. You will enable bulk fuel data management and support data-driven decisions across the DoD enterprise.

Responsibilities include ingest pipelines, COP data integration, data governance, and API development. Strong Spark/Kafka, Databricks, Palantir Ontology experience and DoD familiarity are preferred.