ETL Engineer – TS/SCI w/ Polygraph | Data Pipeline Expert

Core Interface Corporation; Virginia, Northern (IL, KY); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical benefits

401(k)

Paid time off

Parental leave

Tuition assistance

Internet reimbursement

Spot awards

Team events

Job summary

Core Interface Corporation seeks an ETL Engineer to design, build, and optimize data pipelines supporting mission-critical federal programs. You will own the data lifecycle from ingest through delivery in a Linux-based Big Data environment, ensuring robust data flow and documentation.

You will collaborate with data scientists and analysts, apply regex parsing, and leverage ELK, Spark, Airflow, and cloud services to enable scalable analytics and secure data handling in a TS/SCI setting.

Qualifications

• Active TS/SCI with Polygraph
• Bachelor’s degree in Computer Science, Engineering, or related technical field
• Strong Python and/or Java development skills with significant SQL and database experience
• Hands-on experience building ETL pipelines in a Linux-based Big Data environment
• Experience parsing using Regular Expressions
• Experience with ELK Stack
• Familiarity with RESTful API integration and data source consumption
• Clear communication skills in collaborative engineering environments

Responsibilities

• Design, build, and optimize ETL pipelines for large-scale data ingestion and transformation
• Develop custom solutions to ingest, profile, and exploit new and existing data sources
• Collaborate with data scientists and analysts to support model development and analytical workflows
• Maintain comprehensive data layer documentation for the broader engineering team
• Manage all stages of the data lifecycle from ingest planning through display
• Collaborate with teammates, service providers, vendors, and mission users to develop more efficient methods
• Identify and communicate risks, constraints, and tradeoffs in data architecture decisions

Skills

Python

Java

SQL

Regular Expressions

ELK Stack

RESTful API integration

Communication

Education

Bachelor’s degree in Computer Science, Engineering, or related technical field

Tools

Docker

Kubernetes

Apache Airflow

Apache Spark

Databricks

Hadoop

HBase

MapReduce

Job description

Core Interface Corporation seeks an ETL Engineer to design, build, and optimize data pipelines supporting mission-critical federal programs. You will own the data lifecycle from ingest through delivery in a Linux-based Big Data environment, ensuring robust data flow and documentation.

You will collaborate with data scientists and analysts, apply regex parsing, and leverage ELK, Spark, Airflow, and cloud services to enable scalable analytics and secure data handling in a TS/SCI setting.