Senior Applied AI Evaluation Engineer - RAN Ticket Intelligence

The role

We are looking for a hands-on Senior Applied AI Evaluation Engineer to improve how our RAN R&D organization analyzes engineering tickets, supports Root-Cause Analysis, recommends ownership, and learns from resolved cases.

You will build rigorous evaluation datasets, establish meaningful baselines, compare internal and approved external tools, analyze failure modes, and prototype improvements across retrieval, classification, prompting, agent workflows, and model selection.

This role is focused on measurable, evidence-based improvement rather than AI demonstrations. You will assess whether AI-generated conclusions are accurate, grounded in evidence, appropriately calibrated, and useful to engineering teams.

What you will do

Define high-value RAN ticket-intelligence use cases, acceptance criteria, evaluation metrics, and quality guardrails.
Build and maintain representative, versioned evaluation datasets using resolved tickets, Root-Cause Analysis, logs, test evidence, code changes, reviews, reassignment history, and outcomes.
Establish current-tool and non-AI baselines before evaluating new LLM, RAG, search, or agent-based approaches.
Evaluate approved internal, commercial, local, and open-source solutions using secure and reproducible data-handling processes.
Measure retrieval quality, groundedness, diagnosis accuracy, citation support, routing recommendations, calibration, abstention, latency, cost, and human effort.
Design held-out, time-based, edge, and adversarial test cases while preventing data leakage and future-outcome contamination.
Analyze failures and turn incorrect conclusions, misrouting, unsupported claims, and missed evidence into prioritized improvements.
Prototype improvements in search, metadata, context construction, prompting, reranking, classification, agent workflows, and model selection.
Develop reusable evaluation pipelines, tools, services, APIs, dashboards, or documented workflows.
Work closely with AI, RAN, QA, System Integration, Release, Field, data, and engineering teams to review results and support evidence-based decisions.

What you should have

BSc or MSc in Computer Science, Data Science, Machine Learning, Statistics, Electrical Engineering, or a related field, or equivalent practical experience.
5+ years of hands-on experience in applied machine learning, data science, search, natural-language processing, analytics engineering, or AI-enabled software systems.
Recent experience evaluating LLM, RAG, search, or agent systems using representative datasets, task-specific metrics, human review, failure analysis, and regression testing.
Strong Python and SQL skills, with experience building maintainable data pipelines, experiment workflows, services, or analytical tools.
Practical experience with several areas such as information retrieval, embeddings, hybrid search, reranking, classification, structured outputs, tool calling, or common LLM failure modes.
Strong statistical judgment, including sampling, leakage prevention, uncertainty, calibration, precision and recall, temporal drift, and controlled comparison of competing approaches.
Ability to work with semi-structured engineering data from issue-tracking systems, source control, code reviews, continuous integration, logs, dashboards, and test systems.
Clear communication skills and the ability to explain results, limitations, and tradeoffs to technical and business stakeholders.

Nice to have

Knowledge of LTE, 5G NR, Open RAN, telecom-support workflows, or demonstrated ability to learn a technically complex domain through close collaboration with subject-matter experts.
Experience with enterprise search, RAG evaluation, knowledge graphs, process mining, anomaly detection, or graph-based analysis.
Familiarity with Jira, Git or Bitbucket, CI/CD telemetry, software-delivery analytics, evaluation frameworks, experiment tracking, or data versioning.
Experience working with open-weight LLMs, commercial model APIs, local inference, proprietary code, customer logs, or access-controlled engineering data.
Experience handling privacy-sensitive or regulated data in secure enterprise environments.

Parallel Wireless is expanding the ecosystem for Open RAN with the GreenRAN™ energy-efficient Hardware-Agnostic technology. Deployed worldwide, our comprehensive 2G/3G/4G/5G Macro RAN solutions enhance network security while reducing operating expenses. As pioneers of Open RAN, we prioritize innovation, flexibility, and sustainability to help build a more connected, and green networks. Headquartered in the USA with global R&D centers, we are proud to serve over 60 customers worldwide and have been recognized with over 100 industry awards.

Our mission is to accelerate GSMA’s Mobile Net Zero initiative by reducing TCO and driving innovation across the telecom ecosystem.Learn more at www.parallelwireless.com.

Parallel Wireless embraces diversity and equality of opportunity. We are committed to building inclusive and diverse teams representing all backgrounds, with a wide range of perspectives, and empowering industry-leading skills. We welcome and consider applications to join our team from all qualified candidates, regardless of their characteristics. We comply with all applicable laws and regulations on non-discrimination in employment (and recruitment), as well as work authorization and employment eligibility verification requirements.Parallel Wireless does not accept unsolicited resumes or applications from agencies or individuals.

Please do not forward resumes to our jobs alias, Parallel Wireless employees, or any other company location. Parallel Wireless is not responsible for any fees related to unsolicited resumes/applications.

Senior Applied AI Evaluation Engineer - RAN Ticket Intelligence

The role

Senior Applied AI Evaluation Engineer - RAN Ticket Intelligence

The role

What you will do

What you should have

Nice to have

About Parallel Wireless

Similar jobs at other companies

AI Evaluation Scientist

Lead Machine Learning Engineer, Evaluations

Senior AI Engineer - Evaluation & Benchmarking

Manager - Senior AI/ML Engineer

AI Engineer

Principal AI Tool Engineer for Software Automation/Testing

Parallel Wireless