Software Engineer – Ruby (Remote)

Quik Hire Staffing; India; On-site; INR 2,400,000 - 4,000,000; Full time

Job summary

Quik Hire Staffing is seeking a Senior Software Engineer – Ruby to work on LLM evaluation and repository validation in a full‑time capacity. The role focuses on building datasets and tasks from public histories using a synthetic, human‑in‑the‑loop approach, across languages and difficulty levels.

The candidate will analyze open‑source issues, containerize environments with Docker, assess test coverage, and modify Ruby code locally to gauge model performance on bug‑fixing and code‑completion

Qualifications

• Minimum three years of professional software engineering experience with strong proficiency in Ruby and Ruby on Rails.
• Demonstrated expertise with Git, Docker, and basic software pipeline setup for reproducible testing.
• Ability to navigate and understand complex, real‑world codebases, and to run, modify, and test them locally.
• Experience with Ruby testing frameworks (RSpec, Minitest), gem management (Bundler), and code quality tooling.
• Prior contribution to or evaluation of open‑source projects is a plus.

Responsibilities

• Analyze and triage GitHub issues across trending open‑source libraries to identify suitable evaluation scenarios.
• Set up and configure code repositories, including Docker containerization and environment preparation for reproducible testing.
• Evaluate unit test coverage and quality metrics to assess baseline code health and test suite robustness.
• Modify and run Ruby codebases locally to measure LLM performance in bug‑fixing and code‑completion tasks.
• Collaborate with AI researchers to design and select repositories and issues that present challenging cases for LLMs.

Skills

Ruby

Ruby on Rails

Git

Docker

RSpec/Minitest

Tools

Bundler

CI/CD basics

Job description

We are hiring for one of our clients, seeking a Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation) to work on a Full-Time basis. The projects focus on building LLM evaluation and training datasets that address realistic software engineering problems. One approach involves creating verifiable software engineering tasks from public repository histories using a synthetic method with human‑in‑the‑loop, while expanding coverage across programming languages and difficulty levels.

Key Responsibilities

• Analyze and triage GitHub issues across trending open‑source libraries to identify suitable evaluation scenarios.
• Set up and configure code repositories, including Docker containerization and environment preparation for reproducible testing.
• Evaluate unit test coverage and quality metrics to assess baseline code health and test suite robustness.
• Modify and run Ruby codebases locally to measure LLM performance in bug‑fixing and code‑completion tasks.
• Collaborate with AI researchers to design and select repositories and issues that present challenging cases for LLMs.

Required Skills & Qualifications

• Minimum three years of professional software engineering experience with strong proficiency in Ruby and Ruby on Rails.
• Demonstrated expertise with Git, Docker, and basic software pipeline setup for reproducible testing.
• Ability to navigate and understand complex, real‑world codebases, and to run, modify, and test them locally.
• Experience with Ruby testing frameworks (RSpec, Minitest), gem management (Bundler), and code quality tooling.
• Prior contribution to or evaluation of open‑source projects is considered a plus.

More About the Opportunity

This role provides a unique opportunity to work with a global leader in the artificial intelligence industry, contributing to how large language models interact with real code. The work directly influences the development of AI‑assisted software development tools and benchmarks.

Equal Opportunity Employer:

We hire based on skills and expertise. All qualified candidates are welcome regardless of background, experience, or prior employment history. Applications are reviewed solely on demonstrated technical ability and qualifications.