Distributed Systems Engineer – Vector Database(Prototype)

HI Technology & Innovation; Teletrabalho; BRL 120.000 - 160.000; Tempo parcial

Resumo da oferta

A technology firm is seeking an experienced Distributed or Parallel Processing Engineer to develop a lightweight controller layer for a parallelized vector database prototype. The role involves designing a wrapper around an existing system, enabling sharding, and implementing logic for parallel queries. Candidates should have deep expertise in distributed computing and a record of delivering results independently. This is a remote position with a short engagement timeline of about 5 to 10 days.

Qualificações

• Deep expertise in distributed computing and parallel processing.
• Proven track record as a senior engineer able to work independently.
• Strong problem-solving skills with practical experience in building distributed systems.

Responsabilidades

• Design and implement a wrapper/controller layer for the existing vector DB.
• Enable sharding and parallel loading of large vector datasets across instances.
• Implement logic for dispatching and coordinating parallel queries.

Conhecimentos

Distributed computing expertise

Parallel processing

MapReduce design patterns

Problem-solving

Experience with multiple languages

Ferramentas

Python

C#

C

Descrição da oferta de emprego

About the Role

HI Technology & Innovation is collaborating with a pioneering database company to evolve a single-threaded, command-line vector database proof-of-concept, originally written in C#, into a demonstrable, parallelized prototype. We’re looking for an experienced Distributed or Parallel Processing Engineer to lead the development of a lightweight “controller + wrapper” layer that adds multi-instance orchestration, parallelism, and query merging capabilities to the current system.

This is a high-impact, fast-paced role ideal for someone with strong distributed systems thinking and a pragmatic, get-it-done engineering mindset.

What You’ll Tackle

• Design and implement a wrapper / controller layer around the existing single-threaded vector DB POC.
• Enable sharding and parallel loading of large vector datasets across multiple instances.
• Implement logic to dispatch and coordinate parallel queries, aggregate candidate vectors, apply a pluggable distance metric (e.g., Euclidean, cosine similarity, SAD / Manhattan), and merge and sort results before returning to the caller.
• Expose both command-line and REST interfaces to interact with the system.
• Output clear timing metrics to demonstrate speed-ups from parallel execution.
• Deliver a working, testable prototype —production-ready polish is not required.

What We’re Looking For

• Deep expertise in distributed computing and parallel processing, including MapReduce-style design patterns, data sharding, and inter-process communication (IPC).
• Proven track record as a senior engineer who can work independently, move quickly, and deliver results with minimal oversight.
• Strong problem-solving skills and practical experience building distributed systems and prototypes.
• Comfort working across multiple languages and runtimes. Choose whatever tech stack gets the job done fastest (Python, C#, C, etc.).
• Ability to clearly demonstrate and benchmark performance improvements.

Engagement Snapshot

• Timeline: Short-term, estimated 5–10 days of focused work.
• Start Date: ASAP
• Location: Remote
• Deliverables: Fully functional parallel prototype with CLI + REST interfaces, benchmark metrics, and sharded query support.

Ready to build something fast and clever? Apply now and help bring this proof-of-concept to life.