Data Engineer: Real-Time ETL & Streaming Pipelines

Horizon Defense, LLC; Sterling, Northern (VA, KY); Hybrid; USD 110,000 - 150,000; Full time

Benefits offered by this job

Medical, Dental, Vision

401k

Paid Time Off

Holidays

Job summary

Horizon Defense Solutions seeks a Data Engineer to design, develop, and deploy data ingestion and processing pipelines for analytics platforms. You will enable real-time and batch data integration across systems and telemetry streams in a cloud-native environment.

Collaborate with data scientists and software engineers to ensure data is accessible, reliable, and optimized for analytics and AI workflows, with emphasis on scalability and governance.

Qualifications

• Strong understanding of data ingestion patterns and pipeline design.
• Experience with structured and unstructured data sources.
• Experience with streaming architectures and real-time processing.
• Experience with cloud environments (AWS preferred) and data lake/storage (S3).
• Strong communication and collaboration skills.

Responsibilities

• Design and implement ETL/ELT pipelines for structured, semi-structured, and unstructured data sources.
• Develop scalable data ingestion frameworks for batch and real-time data processing.
• Build data connectors and APIs for integrating external systems and operational data sources.
• Support ingestion of high-volume telemetry and instrumentation data streams.
• Normalize and transform incoming data into standardized formats for analytics use.

Skills

Python

SQL

Data ingestion

Streaming architectures

Cloud platforms

Cross-functional collaboration

Education

Bachelor's degree in Computer Science or related field

Tools

Kafka

Kinesis

Docker

Kubernetes

Job description

Horizon Defense Solutions seeks a Data Engineer to design, develop, and deploy data ingestion and processing pipelines for analytics platforms. You will enable real-time and batch data integration across systems and telemetry streams in a cloud-native environment.

Collaborate with data scientists and software engineers to ensure data is accessible, reliable, and optimized for analytics and AI workflows, with emphasis on scalability and governance.