Data Engineer

Arena Technical Resources, LLC

Colorado Springs (CO)

On-site

USD 90,000 - 130,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Arena Technical Resources, LLC is seeking a Data Engineer to build the data foundation for machine-learning applications. You will integrate time-series measurements, including radar and OPIR data, from real-time streams, databases and systems with varying extraction capabilities.

You will collaborate with senior technical staff and the ML team to characterize data, establish reliable pipelines, and deliver cleaned, standardized datasets using Python and AWS.

Qualifications

  • Experience designing, implementing, and operating production streaming and batch data pipelines and their supporting storage systems.
  • Strong software engineering skills, including maintainable code, automated testing, version control, and deployment.
  • Practical understanding of streaming behavior, including filtering, consumer progress, buffering, failure recovery, and replay.
  • Experience organizing time-series, telemetry, sensor, or similarly complex data, with an understanding of schema evolution, partitioning, and efficient access patterns.
  • Demonstrated ability to investigate unfamiliar systems, evaluate technical tradeoffs, and communicate recommendations balancing reliability, performance, maintainability, and cost.

Responsibilities

  • Understand our data sources. Investigate interfaces, formats, coverage, volumes, arrivals, and quality; document findings for capacity planning.
  • Build ingestion pipelines. Develop workflows for streaming and historical data across databases, APIs, files, and interfaces; implement configurable filtering.
  • Shape the data architecture. Evaluate processing, storage, and access options against workloads, reliability, interoperability, and cost; document decisions.
  • Deliver trusted datasets. Establish schemas, timestamps, units, identifiers, and source metadata while preserving meaning; enable access for ML workflows.
  • Operate reliable systems. Build tested, documented pipelines with deployment, monitoring, and access controls; handle duplicates, delays, and reprocessing.

Skills

Streaming pipelines
Batch pipelines
Python
AWS
Data engineering
Version control
Testing
Data modeling
ETL design

Tools

Airflow
Docker
Kubernetes
Terraform

Job description

Job Title: Data Engineer: Streaming & Sensor Data
Job Location: Colorado Springs, CO - Denver, CO, also considered
Eligibility/Clearance: Top Secret

Job Description: We are seeking a Data Engineer to build the data
foundation for our machine learning applications. You will integrate
time-series measurements, including radar and OPIR data, from real-time
streams, application databases, and systems with varying extraction
capabilities. You will collaborate with senior technical staff and the
ML team to characterize available data, establish reliable pipelines,
and deliver cleaned, standardized datasets. You will help select
technologies and develop an architecture based on measured workloads,
application needs, and operational requirements. Our development
environment is primarily Python, with AWS as the preferred cloud
platform.

In this role, you will:
  • Understand our data sources. Investigate available interfaces, message formats, historical coverage, data volumes, arrival rates, and quality limitations. Document findings to guide implementation and capacity planning.
  • Build ingestion pipelines. Develop maintainable workflows for streaming and historical data across databases, APIs, files, and application-specific interfaces. Implement configurable filtering to capture the messages and measurements needed downstream.
  • Shape the data architecture. Evaluate processing, storage, and access options against representative workloads, reliability needs, interoperability, operational complexity, and cost. Document decisions and support the architecture’s evolution as requirements become clearer.
  • Deliver trusted datasets. Establish consistent schemas, timestamps, units, identifiers, and source metadata while preserving measurement meaning. Implement validation and traceability, and define dependable access methods with the ML team. Support future feature, labeling, and training-data workflows.
  • Operate reliable systems. Build tested, documented pipelines with repeatable deployment, monitoring, and appropriate access controls. Handle duplicate, missing, delayed, and out-of-order data, and implement recovery and reprocessing procedures.
We’re looking for someone with:
  • Experience designing, implementing, and operating production streaming and batch data pipelines and their supporting storage systems.
  • Strong software engineering skills, including maintainable code, automated testing, version control, and deployment.
  • Practical understanding of streaming behavior, including filtering, consumer progress, buffering, failure recovery, and replay.
  • Experience organizing time-series, telemetry, sensor, or similarly complex data, with an understanding of schema evolution, partitioning, and efficient access patterns.
  • Demonstrated ability to investigate unfamiliar systems, evaluate technical tradeoffs, and communicate recommendations while balancing reliability, performance, maintainability, and cost.
Additional experience that would be valuable:
  • Python-based data processing and AWS or comparable cloud infrastructure.
  • Radar, OPIR, or other sensor measurement data.
  • Workflow orchestration, containers, infrastructure as code, and automated delivery.
  • Dataset versio

Arena Technical Resources, LLC, (ATR) is an Equal Opportunity Employer (EOE) who will provide equal employment opportunity to employees and applicants for employment without regard to race, ethnicity, religion, color, sex, pregnancy, national origin, age, veteran status, ancestry, sexual orientation, gender identity or expression, marital status, family structure, genetic information, or mental or physical disability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer (Streaming & Sensor Data)
Data Engineer (Streaming & Sensor Data)

Addx Corporation • Colorado Springs (CO)

On-site
USD 90,000 - 130,000
Data Engineer — Streaming & Sensor Data (Top Secret)
Data Engineer — Streaming & Sensor Data (Top Secret)

Arena Technical Resources, LLC • Colorado Springs (CO)

On-site
USD 90,000 - 130,000
Data Engineer
Data Engineer

Addx Corporation • Colorado Springs (CO), Denver (CO)

On-site
USD 120,000 - 150,000
Data Engineer: Streaming & Sensor Data for ML Pipelines
Data Engineer: Streaming & Sensor Data for ML Pipelines

Addx Corporation • Colorado Springs (CO)

On-site
USD 90,000 - 130,000
Senior Data Engineer
Senior Data Engineer

ANAUTICS INC • Oklahoma City (OK)

On-site
USD 90,000 - 120,000
Data Engineer
Data Engineer

Robotic Process Automation LLC • Dallas (TX)

On-site
USD 140,000 - 200,000
Data Engineer
Data Engineer

Quantum Technologies LLC • Dallas (TX)

On-site
USD 130,000 - 190,000
Data Engineer
Data Engineer

Robotics Technologies LLC • Dallas (TX)

On-site
USD 140,000 - 210,000
Data Engineer
Data Engineer

Disruptive Solutions, LLC • Washington

Hybrid
USD 120,000 - 160,000
Data Engineer - Analytics
Data Engineer - Analytics

Air • Pittsburgh

On-site
USD 110,000 - 150,000