Senior Big Data Engineer

QLYS_IN Qualys Security TechServices Private Ltd.; Pune District; On-site; INR 3,200,000 - 5,200,000; Full time

Job summary

Qualys, Inc. is seeking a Senior Big Data Engineer to design, build, and operate core components of the data platform behind the Enterprise TruRisk Platform, processing billions of daily transactions.

You will design pipelines with Spark and Kafka, optimize performance, and mentor junior engineers in a collaborative, security-focused environment. Join a leading cloud security provider and contribute to scalable data infrastructure that supports major enterprise customers worldwide.

Qualifications

• Bachelor's degree in CS/CE or related technical field.
• Minimum 5 years of experience in data engineering or big data platform development.
• Experience designing, building, and optimizing distributed pipelines with Spark and Kafka.

Responsibilities

• Design, build, and maintain high-throughput data platform and pipeline components using Spark and Kafka.
• Debug, tune, and optimize Spark jobs and Kafka pipelines for large data volumes.
• Collaborate with product, professional services, and sales engineering on architecture decisions and trade-offs.
• Establish standards, documentation, and best practices across the engineering team.
• Mentor junior and mid-level engineers on complex technical issues and performance.

Skills

Spark

Kafka

Scala/Java/Python

Mentoring

Distributed pipelines

Education

Bachelor's degree in Computer Science/Engineering

Tools

Apache Spark

Apache Kafka

Job description

Come work at a place where innovation and teamwork come together to support the most exciting missions in the world! Senior Big Data Engineer Job Description: We are seeking a Senior Big Data Engineer to design, build, and operate core components of the data platform and pipeline architecture behind Qualys' Enterprise TruRisk Platform, processing billions of daily transactions.

Responsibilities

• Design, build, and maintain high-throughput data platform and pipeline components using Spark and Kafka.
• Debug, tune, and optimize Spark jobs and Kafka pipelines to manage growing data volumes.
• Collaborate with product, professional services, and sales engineering teams on architecture design and technical trade-offs.
• Establish and refine technical standards, system documentation, and best practices across the engineering team.
• Mentor junior and mid-level engineers on complex technical issues and pipeline performance.

Required Qualifications

• Minimum 5 years of experience in data engineering or big data platform development.
• Bachelor's degree in Computer Science, Computer Engineering, or a related technical field.
• Demonstrated experience designing, building, and optimizing distributed pipelines using Apache Spark and Apache Kafka.
• Hands-on experience scaling data platform infrastructure to process high-volume/high-throughput transaction workloads.

Preferred Qualifications

• Experience working with high-volume, multi-tenant enterprise platform environments.
• Proficiency in Scala, Java, or Python for data processing applications.
• Proven experience mentoring junior or mid-level engineering team members.

Qualys, Inc. (NASDAQ: QLYS) is a pioneer and leading provider of disruptive cloud-based security, compliance and IT solutions with more than 10,000 subscription customers worldwide, including a majority of the Forbes Global 100 and Fortune 100. Qualys helps organizations streamline and automate their security and compliance solutions onto a single platform for greater agility, better business outcomes, and substantial cost savings.