Data Engineer (Cyber Security)

TechDigital Group; Overland Park (KS); On-site; USD 120,000 - 160,000; Full time

Job summary

TechDigital Group is seeking Senior Data Engineers to lead the orchestration and transformation of security telemetry data flows across 100+ sources. You will shape end-to-end architectures, enforce secure data movement, and ensure high-fidelity delivery to downstream analytics platforms.

You will drive schema normalization via OC SF, develop modular, reusable ingestion patterns, and implement data governance with robust security controls.

Qualifications

• 10+ years of experience working in cybersecurity.
• 5+ years on CRIBL/VECTOR/Datadog/Splunk or similar data pipeline platforms.
• 5+ years of scripting: JavaScript, Python, or similar languages.

Responsibilities

• Lead architecture, design, and implementation of scalable data flow pipelines using Cribl and Vector.
• Develop reusable ingestion frameworks for multiple input types and destinations.
• Adopt schema normalization with OC SF, including mapping and validation logic.
• Create data transformations using Groovy, Python, or JavaScript with governance controls.
• Ensure end-to-end data traceability, lineage, and change tracking for audits.
• Collaborate with observability and platform teams for health monitoring and anomaly detection.
• Oversee data integration to ensure high fidelity delivery with minimal loss or drift.
• Lead sessions to evaluate technologies for scaling security telemetry data.

Skills

Cribl

Vector

OCSF

Groovy

Python

JavaScript

Data governance

Data lineage

Security telemetry

SSL/TLS

Tools

Cribl

Vector

Datadog

Splunk

Snowflake

ADX

Log Analytics

Kafka

HTTP

Syslog

Job description

Position Details

• We are seeking Senior Data Engineers to lead efforts in orchestrating and transforming complex security telemetry data flows.
• These individuals will be responsible for high-level architecture and ensuring secure and reliable movement of data between systems, particularly for legacy and non-standard log sources.
• There are 100+ data sources including existing and new that are specific to Cyber Security workloads that are in-scope.
• These tasks will be performed on one or more data ingestion pipelines (Cribl, Vector)

Work Required

• Lead the architecture, design, and implementation of scalable, modular, and reusable data flow pipelines using Cribl, Vector, and other open-source platforms, ensuring consistent ingestion strategies across a complex, multi-source telemetry environment.
• Develop platform-agnostic ingestion frameworks and template-driven architectures to enable reusable ingestion patterns, supporting a variety of input types (e.g., syslog, Kafka, HTTP, Event Hubs, Blob Storage) and output destinations (e.g., Snowflake, Splunk, ADX, Log Analytics, Anvilogic).
• Spearhead the adoption of a schema normalization strategy, leveraging the Open Cybersecurity Schema Framework (OCSF), including field mapping, transformation templates, and schema validation logic designed to be portable across ingestion platforms.
• Design and implement custom data transformations and enrichments using scripting languages such as Groovy, Python, or JavaScript, while enforcing robust governance and security controls (SSL/TLS, client authentication, input validation, logging).
• Ensure full end-to-end traceability and lineage of data across the ingestion, transformation, and storage lifecycle, including metadata tagging, correlation IDs, and change tracking for forensic and audit readiness.
• Collaborate with observability and platform teams to integrate pipeline-level health monitoring, transformation failure logging, and anomaly detection mechanisms.
• Oversee and validate data integration efforts, ensuring high-fidelity delivery into downstream analytics platforms and data stores, with minimal data loss, duplication, or transformation drift.
• Lead technical working sessions to evaluate and recommend best-fit technologies, tools, and practices for managing structured and unstructured security telemetry data at scale.
• Implement data transformation logic including filtering, enrichment, dynamic routing, and format conversions (e.g., JSON CSV, XML, Logfmt) to prepare data for downstream analytics platforms. (100 plus sources of data)
• Contribute to and maintain a centralized documentation repository, including ingestion patterns, transformation libraries, naming standards, schema definitions, data governance procedures, and platform-specific integration details.
• Coordinate with security, analytics, and platform teams to understand use cases and ensure pipeline logic supports threat detection, compliance, and data analytics requirements.

Experience Required:

• 10+ Years of experience working in Cybersecurity
• 5+ Years of experience on CRIBL/Vector/Datadog/Splunk or other data pipeline platforms
• 5+ Years of experience on JavaScript, python, or other scripting language