Data Engineer

Talentify; Whitehall (OH); On-site; USD 131,000 - 171,000; Full time

Job summary

Talentify is seeking a data engineer to architect, build, and operate scalable data pipelines ingesting data from DLA, DoD, and commercial sources. The role focuses on batch and real-time processing using Python, Scala, or Java with Spark and Airflow to support strategic reporting and AI model serving.

Responsibilities include designing data warehouse/data lake structures, implementing data governance, and optimizing cloud warehouses like Snowflake, Redshift, or BigQuery while collaborating with

Qualifications

• U.S. citizen required.
• Experience building scalable data pipelines and cloud data warehouses.
• Proficiency with Python, Scala, or Java.
• Knowledge of CI/CD practices for pipelines.
• Experience with Apache Spark and Airflow.
• Ability to translate data requirements for AI/ML use cases.
• Strong communication to non-technical stakeholders.

Responsibilities

• Architect, develop, test, and maintain scalable data pipelines from diverse sources.
• Develop batch and real-time data solutions using Python/Scala/Java, Spark and Airflow.
• Implement CI/CD for automated testing and deployment of pipelines.
• Design and administer data warehouse and data lake structures (dimensional/Data Vault).
• Optimize cloud data warehouses (Snowflake/Redshift/BigQuery) with partitioning and clustering.
• Enforce data governance and security controls for CUI/Secret data.
• Advise AI/Analytics COE and translate data needs into technical specs.
• Mentor junior staff and conduct peer reviews of data engineering work.
• Explain complex data concepts to leadership to guide decisions.
• Research new technologies and prototype cloud services for modernization.

Skills

Python

Scala

Java

ETL/ELT

Apache Spark

Airflow

CI/CD

Data warehousing

Snowflake

Redshift

BigQuery

Data governance

Mentoring

Communication

Tools

Snowflake

Redshift

BigQuery

Airflow

Spark

Job description

See below for important information regarding this job.
Position will be filled at any of the locations listed below. Site specific salary information as follows:
Battle Creek, MI: $125,776- $163,514
Columbus, OH: $131,245- $170,624
Dayton, OH: $130,461 - $169,604
Fort Belvoir, VA: $143,913- $187,093
New Cumberland, PA: $143,913- $187,093
Ogden, UT: $125,776- $163,514
Philadelphia, PA: $138,595- $180,178
Richmond, VA: $131,385- $170,806

Duties

• Architects, develops, tests, and maintains scalable data pipelines to ingest structured and unstructured data from disparate DLA, DoD, and commercial sources using complex ETL/ELT processes.
• Develops resilient batch and real-time data solutions using Python, Scala, or Java, and frameworks like Apache Spark and Airflow for strategic reporting and operational AI model serving.
• Implements CI/CD practices for automated testing and deployment, troubleshooting pipeline executions to optimize performance, integrity, and cost-effectiveness for DLA systems.
• Leads the design and administration of data warehouse and data lake structures, implementing dimensional and data vault models to support business intelligence and AI/ML use cases.
• Administers and optimizes cloud data warehouses such as Snowflake, Redshift, or BigQuery, managing partitioning, indexing, and clustering strategies for high-performance analytics.
• Enforces data governance, data quality, and enterprise data catalogs, collaborating with security teams to apply access controls, encryption, and data masking for CUI and Secret data.
• Serves as principal technical advisor for the AI and Analytics CoE, translating data requirements from data scientists, ML engineers, and analysts into technical specifications for logistics systems.
• Provides guidance on data architecture best practices, mentors junior technical staff, establishes coding standards, and conducts peer reviews for data engineering projects.
• Communicates complex technical concepts regarding data architecture and pipeline design to non-technical stakeholders and senior leadership to guide strategic organizational decisions.
• Researches, evaluates, and prototypes emerging data technologies and architectural patterns, conducting technical assessments and proofs-of-concept for cloud services and processing engines.
• Leads technical planning and migration strategies to modernize legacy systems, transitioning data workloads to cloud-native platforms while maintaining expertise in industry trends.

Requirements

• Must be a U.S. citizen
• Tour of Duty: Set Schedule
• Security Requirements: Non-Critical Sensitive with Secret Access
• Appointment is subject to the completion of a favorable suitability or fitness determination, where reciprocity cannot be applied; unfavorably adjudicated background checks will be grounds for removal.
• Fair Labor Standards Act (FLSA): Exempt
• Selective Service Requirement: Males born after 12-31-59 must be registered or exempt from Selective Service.
• Recruitment Incentives: Not Authorized
• Bargaining Unit Status: No
• Pre-Employment Physical: Not Required
• Selectees are required to have a REAL ID or other acceptable identification documents to access certain federal facilities. See https://www.tsa.gov/real-id for more information.