Big Data Engineer – AI Security & Threat Analytics

Lumen; Boston (MA); Remote; USD 132,000 - 194,000; Full time

Benefits offered by this job

Health benefits

Life benefits

Voluntary lifestyle benefits

Job summary

Lumen seeks a Big Data Developer to design and optimize large-scale data solutions that identify and disrupt malicious threats across the Internet. You will use Spark, Kafka, Hadoop, and other technologies, leading projects and collaborating with security professionals to protect critical digital infrastructure.

You will work in a remote US-wide team, shaping data architecture, ensuring data quality and security, and mentoring junior developers while exploring PoCs for new capabilities to detect

Qualifications

• Master’s or Bachelor's degree in Computer Science, Engineering, Mathematics, or related field, or equivalent work experience
• Experience in developing data ingestion, data processing and analytical pipelines with big data technologies such as Spark, Kafka, Hadoop, Hive, etc.
• Proficient in programming languages such as Python, Java, Scala, or SQL
• Experience with DevOps practices and tools
• Strong analytical and problem-solving skills, and attention to detail
• Strong communication, leadership, and teamwork skills
• Eager to learn new skills and technologies, and willing to take on challenges

Responsibilities

• Lead the design, development, and implementation of big data solutions that identify and disrupt malicious threats on the Internet.
• Architect and build scalable and reliable data pipelines using Spark, Kafka, Hadoop, and other big data technologies
• Collaborate with cross-functional teams to understand data requirements and provide solutions to achieve business objectives
• Extract, transform, and load data from various sources and formats (relational DBs, APIs, JSON, XML, CSV)
• Ensure data quality, integrity, and security across all data processes
• Develop scalable systems enabling real-time validation of identified resources
• Mentor and guide junior developers fostering continuous learning
• Explore, evaluate, and conduct PoCs to determine new tech solutions to improve threats detection or data processing efficiency

Skills

Python

Java

Scala

SQL

DevOps

Education

Master’s or Bachelor's degree in Computer Science, Engineering, Mathematics, or related field

Tools

Spark

Kafka

Hadoop

Hive

Job description

Lumen seeks a Big Data Developer to design and optimize large-scale data solutions that identify and disrupt malicious threats across the Internet. You will use Spark, Kafka, Hadoop, and other technologies, leading projects and collaborating with security professionals to protect critical digital infrastructure.

You will work in a remote US-wide team, shaping data architecture, ensuring data quality and security, and mentoring junior developers while exploring PoCs for new capabilities to detect