Remote Big Data Engineer - AI Threat Intelligence

Lumen; Frankfort (KY); Remote; USD 154,000 - 194,000; Full time

Benefits offered by this job

Health benefits

Life insurance

Remote-friendly

Job summary

Lumen is seeking a Big Data Developer to build and optimize large-scale data solutions that identify and disrupt threats across the Internet. You’ll deploy Spark, Kafka, Hadoop, and other technologies, collaborating with security professionals to protect critical IT infrastructure.

In this role you’ll lead initiatives, guide peers, and shape data strategy in a remote-friendly environment across the US, delivering real cybersecurity impact while advancing your technical expertise.

Qualifications

• Master’s or Bachelor’s degree in Computer Science, Engineering, Mathematics, or related field, or equivalent work experience
• Prior experience in developing data ingestion, data processing and analytical pipelines with big data technologies, such as Spark, Kafka, Hadoop, Hive, etc.
• Proficient in programming languages, such as Python, Java, Scala, or SQL
• Experience with DevOps practices and tools
• Strong analytical and problem-solving skills, and attention to detail
• Strong communication, leadership, and teamwork skills
• Eager to learn new skills and technologies, and willing to take on challenges

Responsibilities

• Lead the design, development, and implementation of big data solutions that identify and disrupt malicious threats on the Internet
• Architect and build scalable and reliable data pipelines using Spark, Kafka, Hadoop, and other big data technologies
• Collaborate with cross-functional teams to understand data requirements and provide solutions to achieve business objectives
• Extract, transform, and load data from various sources and formats (relational databases, APIs, JSON, XML, CSV)
• Ensure data quality, integrity, and security across all data processes
• Develop scalable systems enabling real-time validation of identified resources
• Mentor and guide junior developers to foster continuous learning and improvement
• Explore, evaluate, and conduct PoC to determine new technology solutions to improve threat detection and data processing

Skills

Big Data

Spark

Kafka

Hadoop

Python

Java

Scala

SQL

DevOps

Leadership

Education

Master’s or Bachelor’s in CS/Engineering/Math or related

Tools

Hive

Kafka

Hadoop

Job description

Lumen is seeking a Big Data Developer to build and optimize large-scale data solutions that identify and disrupt threats across the Internet. You’ll deploy Spark, Kafka, Hadoop, and other technologies, collaborating with security professionals to protect critical IT infrastructure.

In this role you’ll lead initiatives, guide peers, and shape data strategy in a remote-friendly environment across the US, delivering real cybersecurity impact while advancing your technical expertise.