AI Solutions Architect

Remote Jobs; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Broadcastwell LLC is hiring for a remote, full-time role in the United States. The position focuses on reliability and reproducibility of measurement for AI engine evaluations, with ownership of scoring rules and traceable data workflows.

You will design how measurement runs are recorded, prepare datasets for each volume, and decide when results are publishable, documenting any limitations. Flexible hours align with US time zones.

Qualifications

• 5+ years building production systems.
• Proficient in Python, system design, and data pipelines.
• Hands-on experience evaluating large language model outputs.
• A reproducible analysis or evaluation that others can rerun.

Responsibilities

• Own the scoring rules and document them on the methodology page.
• Design measurement run recording so citations trace to answers.
• Turn research requirements into repeatable measurement designs.
• Prepare datasets and analyses for published research volumes.
• Decide when results are solid to publish and note limitations.
• Review client measurements before findings are shared.

Skills

Python

System design

Data pipelines

LLM evaluation

Reproducible analysis

Job description

Open. Remote, United States. Full time.

Broadcastwell measures how AI engines answer buyer questions, and everything the company sells depends on that measurement surviving being checked. This role owns the reliability and reproducibility of the measurement: clear scoring rules, traceable citations and datasets that readers can check.

Responsibilities

• Own the scoring rules applied to every observed answer, and keep them written down on the methodology page.
• Design how each measurement run is recorded so every citation traces back to the answer it came from.
• Turn research requirements into a measurement design the team can repeat on the next volume.
• Prepare the datasets and analysis deposited with each published research volume.
• Decide when a result is solid enough to publish and when a limitation has to be stated beside it.
• Review client measurement before findings reach the client.

What we look for

• Five or more years of experience building production systems.
• Working depth in Python, system design and data pipelines.
• Hands‑on work evaluating large language model output.
• A piece of analysis or evaluation you shipped that someone else could rerun.
• Eligibility to work in the United States without visa sponsorship.

Where and how

• Remote only, hiring in the United States. Relocation is allowed.
• Broadcastwell is a team of consultants and engineers. Sairam Sivakumar, Founder, is accountable for every order.
• Hours are flexible within US time zones, and work is judged on output.

How the work is measured

• Published figures can be recomputed by anyone from the deposited data and analysis code.
• The same question and engine pair scores the same way under the published rules on every run.
• Client findings rest on measurement that holds up when the client checks the receipts.

Pay and terms: $140k to $190k a year, full time, no equity. These are the bands as posted on Wellfound.

Broadcastwell LLC is an equal opportunity employer. Every application is read by a person.