Security Data Engineer

TechDigital Group; Overland Park (KS); On-site; USD 120,000 - 160,000; Full time

Job summary

A technology firm in Overland Park, Kansas is seeking a Senior Security Data Engineer to lead the design and implementation of scalable data pipelines using technologies like Cribl and Apache NiFi. Candidates should have over 10 years of experience in Cybersecurity, with a strong background in data pipeline platforms such as Splunk, and proficiency in scripting languages including JavaScript and Python. This role plays a crucial part in ensuring robust data governance and seamless data integration for operational excellence.

Qualifications

• 10+ years of experience working in Cybersecurity.
• 5+ years of experience on CRIBL/Vector/Datadog/Splunk or other data pipeline platforms.
• 5+ years of experience on JavaScript, Python, or other scripting language.

Responsibilities

• Lead the architecture and implementation of data flow pipelines using Cribl and Apache NiFi.
• Develop data ingestion frameworks to enable reusable patterns.
• Implement data transformations while enforcing governance and security controls.
• Ensure end-to-end traceability and lineage of data.
• Collaborate on pipeline-level health monitoring and anomaly detection.

Skills

Cybersecurity expertise

Data pipeline development

Scripting languages (JavaScript, Python)

Data integration techniques

Tools

Cribl

Apache NiFi

Vector

Splunk

Datadog

Job description

Overview

Senior Security Data Engineer (SIEM Data Pipeline)

Responsibilities

• Lead the architecture, design, and implementation of scalable, modular, and reusable data flow pipelines using Cribl, Apache NiFi, Vector, and other open-source platforms, ensuring consistent ingestion strategies across a complex, multi-source telemetry environment.
• Develop platform-agnostic ingestion frameworks and template-driven architectures to enable reusable ingestion patterns, supporting a variety of input types (e.g., syslog, Kafka, HTTP, Event Hubs, Blob Storage) and output destinations (e.g., Snowflake, Splunk, ADX, Log Analytics, Anvilogic).
• Spearhead the creation and adoption of a schema normalization strategy, leveraging the Open Cybersecurity Schema Framework (OCSF), including field mapping, transformation templates, and schema validation logic—designed to be portable across ingestion platforms.
• Design and implement custom data transformations and enrichments using scripting languages such as Groovy, Python, or JavaScript, while enforcing robust governance and security controls (SSL/TLS, client authentication, input validation, logging).
• Ensure full end-to-end traceability and lineage of data across the ingestion, transformation, and storage lifecycle, including metadata tagging, correlation IDs, and change tracking for forensic and audit readiness.
• Collaborate with observability and platform teams to integrate pipeline-level health monitoring, transformation failure logging, and anomaly detection mechanisms.
• Oversee and validate data integration efforts, ensuring high-fidelity delivery into downstream analytics platforms and data stores, with minimal data loss, duplication, or transformation drift.
• Lead technical working sessions to evaluate and recommend best-fit technologies, tools, and practices for managing structured and unstructured security telemetry data at scale.
• Implement data transformation logic including filtering, enrichment, dynamic routing, and format conversions (e.g., JSON ↔ CSV, XML, Logfmt) to prepare data for downstream analytics platforms. (100 plus sources of data)
• Contribute to and maintain a centralized documentation repository, including ingestion patterns, transformation libraries, naming standards, schema definitions, data governance procedures, and platform-specific integration details.
• Coordinate with security, analytics, and platform teams to understand use cases and ensure pipeline logic supports threat detection, compliance, and data analytics requirements.

Experience & Qualifications

• 10+ Years of experience working in Cybersecurity
• 5+ Years of experience on CRIBL/Vector/Datadog/Splunk or other data pipeline platforms
• 5+ Years of experience on JavaScript, python, or other scripting language