Real-Time Data Integration Engineer (ETL & Governance)

SmartRecruiters, Inc.; Town of Florida, Northern (NY, KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

SOSi seeks a Data Integration Specialist to build scalable data pipelines and ETL workflows, enabling real-time and batch processing for a DoD customer. You will drive data governance, schema alignment, and interoperability across cloud and on-prem environments, partnering with AI/ML engineers for model training.

Required TS/SCI clearance and a bachelor’s degree or five years of equivalent data integration experience; strong experience with NiFi, Databricks, and SQL data warehouses is expected.

Qualifications

• Active TS/SCI clearance is required.
• Experience developing and managing ETL workflows across multiple sources.
• Proficient in data quality and metadata management.

Responsibilities

• Develop and maintain scalable data pipelines for real-time and batch processing.
• Implement ETL workflows to enable data interoperability across cloud and on-premises environments.
• Manage schema alignment, metadata tagging, and data governance.
• Coordinate with AI/ML engineers to meet model training requirements.
• Submit the Data Ingestion & Transformation Report documenting sources and optimizations.

Skills

ETL pipelines

Data integration

SQL databases

Data governance

Cloud environments

API development

Education

Bachelor's degree in Data Science or related field

Five years of data integration experience

Tools

Apache NiFi

Databricks

SQL-based data warehouses

Job description

SOSi seeks a Data Integration Specialist to build scalable data pipelines and ETL workflows, enabling real-time and batch processing for a DoD customer. You will drive data governance, schema alignment, and interoperability across cloud and on-prem environments, partnering with AI/ML engineers for model training.

Required TS/SCI clearance and a bachelor’s degree or five years of equivalent data integration experience; strong experience with NiFi, Databricks, and SQL data warehouses is expected.