Data Engineer: Databricks & DoD Data Platform Expert

Infopact, INC.; United States; On-site; USD 120,000 - 170,000; Full time

Job summary

Infopact, INC. seeks an experienced Data Engineer to support large-scale DoD data and financial management initiatives. Proficient in Databricks, Advana/War Data Platform, Python, PySpark, and SQL to build scalable data pipelines and trusted data products.

The role involves ETL/ELT design, data-quality frameworks, and AWS GovCloud integration to support reconciliations and analytics for mission-critical stakeholders.

Qualifications

• Experience building, managing, and optimizing data pipelines with Databricks.
• Hands-on work with Advana / War Data Platform for DoD data.
• Advanced Python, PySpark, and Pandas skills for data processing.
• Expert SQL abilities including complex queries and performance tuning.
• Familiarity with Delta Lake, Autoloader, and Databricks workflows.
• Experience in ETL/ELT and large-scale Federal Financial Management data.

Responsibilities

• Design, develop, maintain, and optimize scalable data pipelines in Databricks.
• Build and manage ETL/ELT workflows for ingesting, cleansing, transforming, and aggregating data.
• Develop Databricks solutions using Delta Lake, Autoloader, PySpark, SQL, and workflows.
• Implement Bronze/Silver/Gold medallion data architecture concepts.
• Process and analyze Federal Financial Management and GL datasets.
• Support UoT reconciliations, data traceability, and validation controls.
• Design data-quality frameworks with automated validation and exception handling.
• Develop solutions within AWS GovCloud integrating with S3.
• Troubleshoot pipelines, optimize performance, and address bottlenecks.
• Support CI/CD and source-control using GitLab or similar tools.
• Collaborate with analysts, product owners, SMEs, and Government stakeholders.
• Create technical documentation on pipelines, transformations, lineage, and ops procedures.

Skills

Analytical thinking

Troubleshooting

Problem solving

Communication skills

Tools

Databricks

Advana/War Data Platform

Python

PySpark

Pandas

SQL

Delta Lake

Autoloader

Databricks job scheduling

AWS GovCloud

S3

GitLab

CI/CD

Unity Catalog

Job description

Infopact, INC. seeks an experienced Data Engineer to support large-scale DoD data and financial management initiatives. Proficient in Databricks, Advana/War Data Platform, Python, PySpark, and SQL to build scalable data pipelines and trusted data products.

The role involves ETL/ELT design, data-quality frameworks, and AWS GovCloud integration to support reconciliations and analytics for mission-critical stakeholders.