Senior Applied Ai Evaluation Engineer - Ran Ticket Intelligence

Parallelwireless; Maharashtra; On-site; INR 2,800,000 - 5,200,000; Full time

Job summary

Parallel Wireless is seeking a Senior Applied AI Evaluation Engineer to advance how RAN R&D analyzes tickets, performs root-cause analysis, and learns from resolved cases. You will build rigorous evaluation datasets, establish baselines, and prototype improvements across retrieval, prompting, and model selection to support evidence-based decisions.

The role emphasizes measurable improvement over demonstrations, with collaboration across AI, RAN, QA, and engineering teams to deliver robust,

Qualifications

• BSc or MSc in Computer Science, Data Science, ML, Statistics, Electrical Engineering, or equivalent.
• 5+ years in applied ML, data science, NLP, analytics engineering, or AI-enabled software.
• Experience evaluating LLM/RAG/search/agent systems with datasets and metrics.
• Strong Python and SQL skills with data pipelines and tooling experience.
• Knowledge of information retrieval, embeddings, hybrid search, and model failure modes.
• Clear communication to technical and business stakeholders.

Responsibilities

• Define high-value RAN-ticket intelligence use cases and evaluation metrics.
• Build versioned evaluation datasets from tickets, QA logs, and outcomes.
• Establish baselines before evaluating new LLM, RAG, search, or agent approaches.
• Evaluate internal, commercial, local, and open-source solutions securely.
• Measure retrieval quality, groundedness, diagnosis accuracy, and latency.
• Design robust test cases to prevent data leakage and future outcomes contamination.
• Analyze failures and translate them into prioritized improvements.
• Prototype improvements in search, context construction, prompting, and categorization.
• Develop reusable evaluation pipelines, tools, APIs, dashboards, and workflows.
• Collaborate with AI, RAN, QA, and engineering teams to review results.

Skills

Python

SQL

Machine Learning

NLP

Data pipelines

Experimentation

Communication

Education

BSc / MSc in CS / DS / EE

Tools

Jira

Git

CI/CD

Bitbucket

Job description

Parallel Wireless is reimagining mobile networks with innovative, energy-efficient Open RAN solutions. Join us as we lead the future of telecommunications, driving innovation through green and sustainable networks. Learn more about our mission, vision and values

We are looking for a hands‑on Senior Applied AI Evaluation Engineer to improve how our RAN R&D organization analyzes engineering tickets, supports Root-Cause Analysis, recommends ownership, and learns from resolved cases

You will build rigorous evaluation datasets, establish meaningful baselines, compare internal and approved external tools, analyze failure modes, and prototype improvements across retrieval, classification, prompting, agent workflows, and model selection

This role is focused on measurable, evidence-based improvement rather than AI demonstrations. You will assess whether AI-generated conclusions are accurate, grounded in evidence, appropriately calibrated, and useful to engineering teams

As a secondary area of focus, you will analyze engineering workflows at case and team level to identify bottlenecks, handoffs, dependencies, and opportunities for process improvement

What you will do:

• Define high-value RAN ticket-intelligence use cases, acceptance criteria, evaluation metrics, and quality guardrails.
• Build and maintain representative, versioned evaluation datasets using resolved tickets, Root-Cause Analysis, logs, test evidence, code changes, reviews, reassignment history, and outcomes.
• Establish current-tool and non-AI baselines before evaluating new LLM, RAG, search, or agent-based approaches.
• Evaluate approved internal, commercial, local, and open-source solutions using secure and reproducible data-handling processes.
• Measure retrieval quality, groundedness, diagnosis accuracy, citation support, routing recommendations, calibration, abstention, latency, cost, and human effort.
• Design held-out, time-based, edge, and adversarial test cases while preventing data leakage and future-outcome contamination.
• Analyze failures and turn incorrect conclusions, misrouting, unsupported claims, and missed evidence into prioritized improvements.
• Prototype improvements in search, metadata, context construction, prompting, reranking, classification, agent workflows, and model selection.
• Develop reusable evaluation pipelines, tools, services, APIs, dashboards, or documented workflows.
• Work closely with AI, RAN, QA, System Integration, Release, Field, data, and engineering teams to review results and support evidence-based decisions.

What you should have:

• BSc or MSc in Computer Science, Data Science, Machine Learning, Statistics, Electrical Engineering, or a related field, or equivalent practical experience.
• 5+ years of hands-on experience in applied machine learning, data science, search, natural-language processing, analytics engineering, or AI-enabled software systems.
• Recent experience evaluating LLM, RAG, search, or agent systems using representative datasets, task-specific metrics, human review, failure analysis, and regression testing.
• Strong Python and SQL skills, with experience building maintainable data pipelines, experiment workflows, services, or analytical tools.
• Practical experience with several areas such as information retrieval, embeddings, hybrid search, reranking, classification, structured outputs, tool calling, or common LLM failure modes.
• Strong statistical judgment, including sampling, leakage prevention, uncertainty, calibration, precision and recall, temporal drift, and controlled comparison of competing approaches.
• Ability to work with semi-structured engineering data from issue-tracking systems, source control, code reviews, continuous integration, logs, dashboards, and test systems.
• Clear communication skills and the ability to explain results, limitations, and tradeoffs to technical and business stakeholders.

Preferred qualifications:

• Knowledge of LTE, 5G NR, Open RAN, telecom-support workflows, or demonstrated ability to learn a technically complex domain through close collaboration with subject-matter experts.
• Experience with enterprise search, RAG evaluation, knowledge graphs, process mining, anomaly detection, or graph-based analysis.
• Familiarity with Jira, Git or Bitbucket, CI/CD telemetry, software-delivery analytics, evaluation frameworks, experiment tracking, or data versioning.
• Experience working with open-weight LLMs, commercial model APIs, local inference, proprietary code, customer logs, or access-controlled engineering data.
• Experience handling privacy-sensitive or regulated data in secure enterprise environments.

Parallel Wireless is expanding the ecosystem for Open RAN with the GreenRAN™ energy-efficient Hardware-Agnostic technology. Deployed worldwide, our comprehensive 2G/3G/4G/5G Macro RAN solutions enhance network security while reducing operating expenses. As pioneers of Open RAN, we prioritize innovation, flexibility, and sustainability to help build a more connected, and green networks. Headquartered in the USA with global R&D centers, we are proud to serve over 60 customers worldwide and have been recognized with over 100 industry awards. Our mission is to accelerate GSMA’s Mobile Net Zero initiative by reducing TCO and driving innovation across the telecom ecosystem. Learn more at www.parallelwireless.com

Parallel Wireless embraces diversity and equality of opportunity. We are committed to building inclusive and diverse teams representing all backgrounds, with a wide range of perspectives, and empowering industry-leading skills. We welcome and consider applications to join our team from all qualified candidates, regardless of their characteristics. We comply with all applicable laws and regulations on non-discrimination in employment (and recruitment), as well as work authorization and employment eligibility verification requirements. Parallel Wireless does not accept unsolicited resumes or applications from agencies or individuals. Please do not forward resumes to our jobs alias, Parallel Wireless employees, or any other company location. Parallel Wireless is not responsible for any fees related to unsolicited resumes/applications.