Big Data Engineer - Threat Detection & Security (Remote)

Lumen; Denver (CO); Remote; USD 139,000 - 185,000; Full time

Job summary

Lumen, a trusted network leader for AI-ready connectivity, seeks a Big Data Developer to design and optimize large-scale data solutions for identifying and disrupting malicious threats across the Internet. This remote position invites candidates from across the US to join a mission-driven team.

You will architect scalable data pipelines with Spark, Kafka, Hadoop, and Hive, collaborate with security engineers, and ensure data quality and security across processes while mentoring junior developers.

Qualifications

• Master’s or Bachelor's degree in Computer Science, Engineering, Mathematics, or related field, or equivalent work experience
• Prior experience in developing data ingestion, data processing and analytical pipelines with big data technologies, such as Spark, Kafka, Hadoop, Hive, etc.
• Proficient in programming languages, such as Python, Java, Scala, or SQL
• Experience with DevOps practices and tools
• Strong analytical and problem‑solving skills, and attention to detail
• Strong communication, leadership, and teamwork skills
• Eager to learn new skills and technologies, and willing to take on challenges

Responsibilities

• Lead the design, development, and implementation of big data solutions that allow the organization to identify and disrupt malicious threats on the Internet.
• Architect and build scalable and reliable data pipelines using technologies like Spark, Kafka, Hadoop, and other big data technologies
• Collaborate with cross functional teams to understand data requirements and provide solutions to achieve business objectives
• Extract, transform, and load data from various sources and formats, such as relational databases, APIs, JSON, XML, CSV, etc.
• Ensure data quality, integrity, and security across all data processes
• Develop scalable systems that enable real‑time validation of identified resources
• Mentor and guide junior developers fostering a culture of continuous learning and improvement
• Explore, evaluate, and conduct PoC to determine new technology solutions that would improve our ability to detect newer threats or add efficiency to current data processing pipelines

Skills

Spark

Kafka

Hadoop

Hive

Python

Java

Scala

SQL

DevOps

Leadership

Education

Master's or Bachelor's degree in Computer Science, Engineering, Mathematics, or related field

Tools

DevOps tooling

Job description

Lumen, a trusted network leader for AI-ready connectivity, seeks a Big Data Developer to design and optimize large-scale data solutions for identifying and disrupting malicious threats across the Internet. This remote position invites candidates from across the US to join a mission-driven team.

You will architect scalable data pipelines with Spark, Kafka, Hadoop, and Hive, collaborate with security engineers, and ensure data quality and security across processes while mentoring junior developers.