Senior Data Engineer with Security Clearance

Castalia Systems; Arlington (VA); Hybrid; USD 150,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, vision coverage

401k matching

Generous PTO

Paid holidays

Professional training opportunities

Pet insurance

Job summary

Castalia Systems is seeking a senior data engineer to design, develop, and maintain scalable data pipelines and ETL processes across multiple sources. You will ensure data quality, reliability, and performance while collaborating with software engineers, data scientists, and mission stakeholders to support AI/ML analytics.

The role requires 8+ years of experience, BS degree, and proficiency in Python, Databricks, and relational databases.

Qualifications

• BS Degree.
• 8+ years of professional experience.
• Proficient in Python for data manipulation and scripting.
• Databricks experience.
• Proficient in designing and implementing ETL data processes.
• Experience with at least one relational database (PostgreSQL/MySQL/SQLite).
• Experience with Agile SCRUM methodologies.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes to ingest, transform, integrate, and deliver mission-critical data from multiple sources.
• Ensure data availability, quality, reliability, and performance by monitoring data workflows and optimizing processing and storage.
• Develop data solutions using Python, Databricks, SQL, and relational databases.
• Collaborate with Software Engineers, Data Scientists, and stakeholders to support AI/ML applications and analytics.
• Design and implement secure data integration and transfer solutions in classified environments.

Skills

Python

Databricks

ETL

SQL

PostgreSQL

MySQL

SQLite

Agile SCRUM

Education

BS Degree

Tools

Databricks

PostgreSQL

MySQL

SQLite

NiFi

Qlik

NoSQL

Graph Databases

Job description

Job Type: Full- Time Workplace Type: Hybrid Clearance: Active TS/SCI Must be a U.S. Citizen

Roles and Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes to ingest, transform, integrate, and deliver mission-critical data from multiple sources.
• Ensure data availability, quality, reliability, and performance by monitoring data workflows, troubleshooting issues, and optimizing data processing and storage solutions.
• Develop and maintain data solutions using Python, Databricks, SQL, relational databases, and other modern data engineering technologies.
• Collaborate with Software Engineers, Data Scientists, and mission stakeholders to support the development and deployment of AI/ML applications and data-driven analytics.
• Design and implement secure data integration and transfer solutions while supporting database architecture, data organization, scalability, and cybersecurity requirements within classified environments.

Required Qualifications

• BS Degree.
• 8+ years of professional experience needed for MOST of our opportunities.
• Python experience with data manipulation, scripting, and more.
• Databricks experience.
• Proficient in designing and implementing Extract Transform Load (ETL) data processes.
• Experience with at least ONE relational database, such as PostgreSQL, MySQL or SQLite.
• Experience with Agile SCRUM methodologies.

Preferred Qualifications

• Familiar with database design, server management, connection protocols, database normalization, and principles of information theory.
• Experience with data cleaning or data organization.
• Familiar with NiFi, SQL, NoSQL and Graph Databases and Data Lakes.
• Familiar with Dashboarding tools similar to Qlik.
• Familiar with Cybersecurity with design and implementation of unique data transfer protocols.

Physical Requirements/Work Environment

• ONSITE 3-5 days per week.
• Normal office environment. Travel

Company Description

Castalia Systems is a proven business partner providing mission critical solutions to the Federal Government. We provide cutting edge solutions from Securing and Managing Data to Systems Engineering and Development. Castalia Systems is a pioneer in Artificial Intelligence Design and Application. With our vast knowledge of our customers’ needs and relevant technology, our team is able to bring successful solutions to every mission. We are one-upping our competitors by providing premium IT solutions and platforms with cutting-edge technology so it’s so evident when you compare us with anyone.

Compensation

• Medical, dental, vision coverage, 401k matching, generous PTO, paid holidays, professional training opportunities, and even pet insurance to ensure your furry friends are cared for too.
• All regularly scheduled employees working at least 30 hours per week are eligible to participate in Castalia Systems’ benefit programs. Individuals that do not meet the threshold are only eligible for select offerings, not inclusive of health benefits.
• The projected compensation range for this position is $150,000.00 to $160,000.00 (annualized USD).

Disclaimer

Castalia Systems is an equal employment opportunity and affirmative action employer and strives to comply with all applicable laws prohibiting discrimination based on race, color, creed, sex, sexual orientation, age, national origin, or ancestry, physical or mental disability, veteran status, marital status, HIV-positive status, as well as any other category protected by federal, state, or local laws. All such discrimination is unlawful, and all persons involved in the operations of the company are prohibited from engaging in this type of conduct.