Senior Distributed Systems Engineer

Qualys; Foster City (CA); On-site; USD 180,000 - 250,000; Full time

Job summary

Qualys is seeking a Senior Software Engineer to advance our data platform and middleware. You will prototype, design, and build a scalable SaaS data platform handling vast transactions and terabytes of data daily. Expect to scale ClickHouse, optimize ingestion and query performance, and drive automated deployment pipelines.

You will collaborate with a talented team, contribute to open source, and ensure resilient operations across cloud and on-prem environments.

Qualifications

• BS/MS in Computer Science or related field.
• 5+ years of SaaS/ cloud software experience.
• Proven expertise in ClickHouse internals and architecture.
• Experience with public clouds (AWS, GCP, Azure) or private clouds.
• Strong problem-solving and debugging abilities.
• Hands-on coding with high-quality, production-ready APIs.

Responsibilities

• Prototype, design, develop and support a scalable SaaS data platform.
• Operate and scale ClickHouse on premise or in cloud environments.
• Drive capacity planning and performance tuning for big data workloads.
• Deliver resilient client SDKs and engage with open-source community.
• Instrument day 1 and day 2 operations for data technologies; enable zero-downtime scaling.
• Collaborate across teams to meet reliability, cost, and availability goals.

Skills

SaaS software development

Problem-solving

Distributed systems

Agile software development

Data structures & algorithms

Architecting scalable systems

Strong debugging skills

Excellent communication

Education

BS/MS in Computer Science or related field

Tools

ClickHouse

Terraform

Docker

Kubernetes

CI/CD pipelines

Maven/Gradle

Grafana/Prometheus

Kafka/Elasticsearch/Cassandra

HashiCorp tools (Consul, Vault, etc.)

Job description

We are seeking a highly motivated and talented Senior Software Engineer to work on Qualys’ next-generation Data Platform and middleware.

Working alongside a very talented team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable SaaS based cloud security Data Platform. This is a great opportunity to be an integral part of a team building Qualys’ next generation microservices based technology platform processing over a 100 million transactions and terabytes of data per day, leverage open source technologies, and work on challenging and business-impacting projects.

Responsibilities

For this position, we are especially looking for someone with below capabilities:

• Brings deep expertise in ClickHouse internals and architecture, various ClickHouse engines, partitioning, ordering keys, indexing, and query execution.
• Design and review schemas against real-world read/write patterns, optimizing ingestion, query performance, and storage efficiency.
• Operate ClickHouse at scale on premise or cloud, including replication, sharding, workload management, performance tuning, and production troubleshooting.
• Drive capacity planning across CPU, memory, storage I/O, and network resources, accounting for workload growth, concurrency, background merges, and reliability requirements.
• You will be responsible for engineering of services running data stored with millions of transactions per second, scaling to 100s of terabytes of data. You will be responsible for driving the data source and providing data platform as a service to various applications built in Qualys.
• Be the face and developer advocate of the technology stack you own and contribute back to open-source community if any enhancement needed to fit in Qualys ecosystem.
• You will be instrumenting both day 1 and day 2 operations on these data technologies keeping the 100s of thousands if not millions of operations per second. Migrate, burst into cloud and scale the stack with zero or least amount of downtime.
• You will be responsible for delivering resilient client SDKs to the applications to communicate with the data source you are responsible for.
• You will be constantly in the hunt for better solutions out there to resolve the current scaling, availability, and cost-effective alternative solutions.

Qualifications

• BS/MS degree in Computer Science, Applied Math or related field.
• Experience and expertise in ClickHouse as outlined earlier
• 5+ years hands-on experience with developing SaaS based software products and solutions for service-oriented deployments in public (AWS, GCP, Azure, OCI, etc.) or private clouds.
• Proven expertise in data-driven critical thinking problem-solving
• Ability to debug systems of software and people
• Demonstrated ability to write high-quality, production-ready code (readable, well-tested, with well-designed APIs)
• Demonstrated passion for innovation with an ability to work within agile software development methodologies
• Experience with infrastructure orchestration and automation tools (Terraform)
• Hands-on experience using CI/CD pipelines
• Strong analytical and debugging skills.
• Knowledge of JVM concepts like garbage collection, heap, stack, profiling, class loading, etc.
• Strong in data structures and algorithms.
• Ability to clearly articulate and communicate technical concepts within and across teams.
• Knowledge of the below is a plus
• Experience with distributed systems like Cassandra, Redis, Elasticsearch, Kafka, Ceph, Flink, Spark or similar technologies
• Experience with applying data encryption and data security standards.
• Experience with container and orchestration technologies such as Docker, Kubernetes etc.
• Experience with monitoring tools such as Graphite, Grafana and Prometheus.
• Experience with HashiCorp technologies such as Consul, Vault, Terraform and Vagrant.
• Experience with configuration management tools such as Chef, Puppet or Ansible.
• In-depth experience with continuous integration and continuous deployment pipelines.
• Exposure to Maven, Ant or Gradle for builds.