Data Engineer – Top Secret Clearance

Metric5; United States; On-site; USD 140,000 - 170,000; Full time

Job summary

Metric5 is seeking a Data Engineer to design and maintain scalable data infrastructure across client domains. You will build real-time streaming pipelines, CDC patterns and ensure secure, reliable data delivery using AWS/OpenShift stack.

Collaboration with backend teams and data scientists is essential. You will leverage Kafka, Flink, OpenSearch and vector AI features to enrich data workflows, implement robust ingestion pipelines, and optimize search capabilities for mission-critical

Qualifications

• 7 years of experience building real‑time data streaming architectures and CDC patterns.
• Hands-on with Apache Kafka and complex stream processing in Apache Flink.
• Extensive experience with OpenSearch/Elasticsearch: ingest pipelines and index templates.
• Experience integrating AI/ML models into data pipelines and vector databases.
• Proven Kubernetes/OpenShift deployment with IaC using Helm and ArgoCD.

Responsibilities

• Design, build and maintain CDC pipelines from MySQL to Kafka using Debezium.
• Develop Flink apps to process streams, join multi-table data and emit denormalized records.
• Configure Vector pipelines and securely sink data into OpenSearch with custom ingest pipelines.
• Architect OpenSearch data management workflows and ensure high availability.
• Collaborate with NestJS developers to define data contracts and query patterns.

Skills

Kafka

Flink

OpenSearch/Elasticsearch

Debezium

Kubernetes/OpenShift

Helm

ArgoCD

NodeJS/NestJS

Vector databases

AI/ML in data pipelines

Education

Bachelor's Degree

Tools

OpenShift

Helm

ArgoCD

Datadog Vector

Kafka tooling

Job description

Location

Springfield VA On site 2 days a week Clearance Top Secret Required

Position Overview

As a Data Engineer you will work closely with O&M development product design and client teams to maintain enhance and deliver secure data infrastructure and search capabilities across client domains The role requires experience building scalable real time data streaming architectures designing CDC pipelines and ensuring data is accessible reliable and mission ready You will help build modernize and sustain data workflows using AWS Cloud OpenShift and related technologies while supporting secure deployment system performance and continuous improvement

Responsibilities

Design build and maintain robust Change Data Capture (CDC) pipelines extracting data from MySQL databases using Debezium and routing to Apache Kafka Develop and optimize Apache Flink applications to consume Debezium topics perform complex multi table joins and output denormalized records back to Kafka Configure and manage Vector pipelines to consume Flink processed Kafka topics perform object remapping and reliably sink data into OpenSearch Architect OpenSearch data management workflows including the design and implementation of custom ingest pipelines index templates and lifecycle policies Leverage AI skills to enhance data enrichment processes implement vectorsemantic search capabilities within OpenSearch and support advanced analytics Deploy scale and maintain data infrastructure on OpenShift using Helm charts and ArgoCD following GitOps best practices Monitor system health tune performance across the entire data streaming lifecycle Kafka Flink OpenSearch and ensure high availability and fault tolerance Collaborate with backend engineers to ensure OpenSearch indexes are highly optimized for performant querying by downstream NestJS applications Perform root cause analysis for system application data pipeline and end user issues assisting Tier 2 support teams with complex problem resolutions Maintain technical documentation related to data architecture data flows APIs system configurations and operational procedures while supporting system security coordination Support occasional after hours and weekend work for operational issue resolution production deployments data migrations and maintenance windows

Required Skills

7 years of experience building scalable real time data streaming architectures and CDC patterns Hands on experience with Apache Kafka and deep proficiency in writing complex stream processing jobs using Apache Flink Extensive experience with OpenSearch or Elasticsearch including cluster management writing ingest pipelines managing index templates and writing complex optimized search queries Applied knowledge of integrating AIML models into data pipelines working with vector databases eg OpenSearch k NN or building AI driven data products Experience with Debezium for CDC and Vector by Datadog for observability and data routing Proven experience deploying applications in KubernetesOpenShift environments with strong familiarity with infrastructure as code and deployment workflows using Helm and ArgoCD Ability to work closely with software engineers particularly those using NodejsNestJS to define data contracts and query patterns Ability to design develop and operate highly available data services across availability zones and regions Self starter with strong problem solving analytical decision making and verbal and written communication skills

Preferred Skills

Experience working with cloud platforms such as AWS Support for data quality data validation metadata management and data governance practices Familiarity with secure software development practices vulnerability remediation access control and compliance requirements in federal or classified environments Familiarity with Agile Scrum SAFe or other iterative development methodologies with experience delivering solutions to government customers Experience serving in an on call role supporting emergency response to application or system issues on occasion Certifications Security certification is preferred Other relevant certifications include CCNA CCNP CISA CISSP and CISM

Years of Experience

5 years

Education

Bachelors Degree

Salary

140000 170000

About Metric5

Metric5 is a small business with big company benefits We have a passionate team of smart fun caring professionals and we are here for the long haul Join our growing team in a business where your contributions make an enormous impact Our organization offers a comprehensive employee benefits package continuous professional development with a best in class company culture that is enjoyable to work in and supports the growth of each of our professionals Our benefits include Health & Dental Insurance with 100 of individual coverage paid for by the companyVision InsuranceLife & Short Term Disability Insurance401K with company match employees are immediately vestedPaid Vacation9 Paid Holidays per year plus 2 paid floating holidaysParental LeaveEmployee BonusesProfessional Development Reimbursement ProgramTuition Assistance ProgramMetric5 is an Equal Opportunity Employer All qualified applicants will receive consideration for employment without regard to race color religion sex sexual orientation gender identity national origin disability or status as a protected veteran