Senior Data Engineer - TS/SCI Hybrid, Secure Data Pipelines

Castalia Systems; Arlington (VA); Hybrid; USD 150,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, vision coverage

401k matching

Generous PTO

Paid holidays

Professional training opportunities

Pet insurance

Job summary

Castalia Systems is seeking a senior data engineer to design, develop, and maintain scalable data pipelines and ETL processes across multiple sources. You will ensure data quality, reliability, and performance while collaborating with software engineers, data scientists, and mission stakeholders to support AI/ML analytics.

The role requires 8+ years of experience, BS degree, and proficiency in Python, Databricks, and relational databases.

Qualifications

• BS Degree.
• 8+ years of professional experience.
• Proficient in Python for data manipulation and scripting.
• Databricks experience.
• Proficient in designing and implementing ETL data processes.
• Experience with at least one relational database (PostgreSQL/MySQL/SQLite).
• Experience with Agile SCRUM methodologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes to ingest, transform, integrate, and deliver mission-critical data from multiple sources.
• Ensure data availability, quality, reliability, and performance by monitoring data workflows and optimizing processing and storage.
• Develop data solutions using Python, Databricks, SQL, and relational databases.
• Collaborate with Software Engineers, Data Scientists, and stakeholders to support AI/ML applications and analytics.
• Design and implement secure data integration and transfer solutions in classified environments.

Skills

Python

Databricks

ETL

SQL

PostgreSQL

MySQL

SQLite

Agile SCRUM

Education

BS Degree

Tools

Databricks

PostgreSQL

MySQL

SQLite

NiFi

Qlik

NoSQL

Graph Databases

Job description

Castalia Systems is seeking a senior data engineer to design, develop, and maintain scalable data pipelines and ETL processes across multiple sources. You will ensure data quality, reliability, and performance while collaborating with software engineers, data scientists, and mission stakeholders to support AI/ML analytics.

The role requires 8+ years of experience, BS degree, and proficiency in Python, Databricks, and relational databases.