Top Secret Data Engineer — Real-Time Data & OpenSearch

Metric5; United States; On-site; USD 140,000 - 170,000; Full time

Job summary

Metric5 is seeking a Data Engineer to design and maintain scalable data infrastructure across client domains. You will build real-time streaming pipelines, CDC patterns and ensure secure, reliable data delivery using AWS/OpenShift stack.

Collaboration with backend teams and data scientists is essential. You will leverage Kafka, Flink, OpenSearch and vector AI features to enrich data workflows, implement robust ingestion pipelines, and optimize search capabilities for mission-critical

Qualifications

• 7 years of experience building real‑time data streaming architectures and CDC patterns.
• Hands-on with Apache Kafka and complex stream processing in Apache Flink.
• Extensive experience with OpenSearch/Elasticsearch: ingest pipelines and index templates.
• Experience integrating AI/ML models into data pipelines and vector databases.
• Proven Kubernetes/OpenShift deployment with IaC using Helm and ArgoCD.

Responsibilities

• Design, build and maintain CDC pipelines from MySQL to Kafka using Debezium.
• Develop Flink apps to process streams, join multi-table data and emit denormalized records.
• Configure Vector pipelines and securely sink data into OpenSearch with custom ingest pipelines.
• Architect OpenSearch data management workflows and ensure high availability.
• Collaborate with NestJS developers to define data contracts and query patterns.

Skills

Kafka

Flink

OpenSearch/Elasticsearch

Debezium

Kubernetes/OpenShift

Helm

ArgoCD

NodeJS/NestJS

Vector databases

AI/ML in data pipelines

Education

Bachelor's Degree

Tools

OpenShift

Helm

ArgoCD

Datadog Vector

Kafka tooling

Job description

Metric5 is seeking a Data Engineer to design and maintain scalable data infrastructure across client domains. You will build real-time streaming pipelines, CDC patterns and ensure secure, reliable data delivery using AWS/OpenShift stack.

Collaboration with backend teams and data scientists is essential. You will leverage Kafka, Flink, OpenSearch and vector AI features to enrich data workflows, implement robust ingestion pipelines, and optimize search capabilities for mission-critical