Big Data Engineer - Threat Detection (Remote)

Lumen Technologies, Inc.; United States; Remote; USD 132,000 - 194,000; Full time

Job summary

Lumen Technologies, Inc. is seeking a Big Data Developer (Remote) to design and optimize large-scale data solutions that detect and disrupt threats. You will build scalable pipelines with Spark, Kafka, Hadoop, and Hive, and collaborate across teams to transform data into actionable insights.

You’ll mentor juniors, evaluate new technologies, and contribute to real-time validation systems while advancing cybersecurity capabilities across a remote US team.

Qualifications

• Master’s or Bachelor's degree in CS/Engineering/Math or equivalent work experience.
• Experience building data ingestion and processing pipelines with Spark, Kafka, Hadoop, Hive.
• Proficiency in Python, Java, Scala, or SQL.
• Familiar with DevOps practices and tools.
• Strong analytical, problem-solving, and teamwork skills.

Responsibilities

• Lead design, development, and implementation of big data solutions to identify threats on the Internet.
• Architect and build scalable data pipelines using Spark, Kafka, Hadoop and related tech.
• Collaborate with cross‑functional teams to meet data requirements and business goals.
• Extract, transform, and load data from diverse sources (RDBMS, APIs, JSON, XML, CSV).
• Ensure data quality, integrity, and security across processes.
• Develop real-time validation systems and mentor junior developers.
• Evaluate new tech via PoCs to improve threat detection and data processing.

Skills

Spark

Kafka

Hadoop

Hive

Python

Java

Scala

SQL

DevOps practices

Leadership

Communication

Education

Master’s or Bachelor’s degree in Computer Science, Engineering, Mathematics, or related field

Tools

DevOps tools

Job description

Lumen Technologies, Inc. is seeking a Big Data Developer (Remote) to design and optimize large-scale data solutions that detect and disrupt threats. You will build scalable pipelines with Spark, Kafka, Hadoop, and Hive, and collaborate across teams to transform data into actionable insights.

You’ll mentor juniors, evaluate new technologies, and contribute to real-time validation systems while advancing cybersecurity capabilities across a remote US team.