RAG in GxP: the answer is only one part of the evidence

The signal: retrieval needs controls of its own

The ISPE programme includes controls for retrieval-augmented GenAI in GxP. Retrieval-augmented generation, often called RAG, connects a language model to selected information sources. Our practical interest is the chain that turns a user question into a supported answer, and the places where that chain can lose scientific context. [1]

Adding references to a response does not establish that the referenced record was appropriate for the user, current for the task or sufficient for the conclusion. A useful evaluation therefore examines retrieval, interpretation and presentation separately. The application should help a reviewer recognize when those components disagree.

Knowledge management depends on how the organization maintains knowledge

A Pharma 4.0 knowledge workflow includes the people who create, approve, withdraw and interpret information. A retrieval index is one component. Source ownership, effective-version rules, training, review expectations and escalation determine whether an answer can be used in the real process. [4]

The librarian or document owner maintains source status; the scientist interprets the process context; the system owner manages access and refresh; the receiving reviewer decides whether the evidence answers the question. Track unresolved questions as learning about the knowledge process rather than rewarding the assistant for producing an answer every time.

Controlled knowledge is an organizational capability. Retrieved evidence, scientific calculations and generated interpretation need distinct support and an accountable receiving reviewer.THE BIOMEDANA LENS

A hypothetical method question reveals the version problem

Imagine a scientist asking which sample preparation applies to a specific analytical run. The library contains a current method, an archived method and a development note discussing an alternative preparation. All three use similar terminology. A fluent answer assembled from the wrong combination could look coherent while failing the actual question.

The retrieval request needs context: product, process stage, method identity, effective version and the purpose of the question. If the historical run used an earlier authorized method, the current method may be the wrong basis for describing what happened. Current knowledge and historical evidence require an explicit relationship rather than an assumption that the newest document is always the answer.

Establish a controlled source set

Our proposed source register records ownership, document identity, version, effective status, access restrictions and the reason a source belongs in the retrieval scope. It also describes ingestion and update behavior. A source should not become authoritative merely because it was easy to load or similar to a trusted document.

Changes need to propagate deliberately. Decide when a withdrawn document stops appearing in operational answers and how retained historical answers remain interpretable. A cached passage can outlive the source state that justified its use. Testing document updates should therefore include the retrieval index, cached responses and any exported answer packet. Drug CGMP data-integrity guidance supplies relevant recordkeeping background. [2]

A source version is only part of the retrieval record

The source can be correct while its representation is wrong. A parser may lose a footnote, a table may separate a value from its unit, or the retrieved passage may omit the condition that makes an instruction applicable. A missing source-to-chunk link can make an apparently relevant answer impossible to reconstruct.

For a controlled retrieval workflow, retain the source set and relevant versions of ingestion, chunking, embedding, index and retrieval configuration, together with the access context and evidence used for the answer. A change in one dependency should trigger an impact assessment proportionate to the task, rather than an assumption that the language model is the only component that matters.

Challenge the workflow with a table, an obsolete method, an unauthorized source and a conflict between applicable records. Test answer support, completeness, permission enforcement and appropriate refusal separately. Similarity scores or an automated evaluator can help organize review; they do not by themselves establish scientific correctness.

Test whether the source supports the sentence

Build a small evaluation set around real user tasks. Include a question with no sufficient answer, a question requiring two records and a question where the tempting passage concerns another product. For each response, ask whether the cited material supports the specific claim, not simply whether its topic looks related.

Retain an explicit distinction between what the source states and what the system infers. A reviewer may accept a factual excerpt while rejecting the proposed operational implication. When a response synthesizes several records, make those joins inspectable. A missing qualification in a short answer can be more consequential than a minor wording error in a long summary.

Treat permissions as part of retrieval correctness

Run the same question as users with different project access. Verify that restricted content does not appear through passages, titles, summaries or exports. The authorization boundary should apply to the complete answer path, not just to whether the original file link opens when clicked.

Include an untrusted document that contains instructions to ignore restrictions or retrieve another study. That material is evidence to inspect, not authority to change application policy. The system should preserve the intended task and avoid adopting commands found in retrieved content. Our proposed test focuses on behavior; a general statement that the model is secure is not a result.

Define what happens when the answer should stop

The application needs a useful insufficient-evidence response. It should explain which context is missing, identify the relevant owner or source class and retain the attempted question without inventing an operational instruction. A refusal that offers no route to resolve the issue can push users into informal workarounds.

Measure answer support, retrieval omissions, inappropriate source use, permission failures and reviewer correction time. Choose acceptance criteria for the actual task before testing. A generic answer-accuracy percentage can hide a rare but consequential failure involving a restricted record or an effective-version question. Report performance by the scenarios that matter.

BioMedAna: distinguish a retrieved answer from a computed scientific result

In BioMedAna, retrieval and scientific computation have distinct roles. A knowledge answer can cite a method or a historical record. A numerical scenario can invoke the physics-based engineering layer, reaction-kinetics service or a suitable mechanistic/hybrid model. The agent coordinates these functions and explains their outputs; retrieval by itself does not calculate a mass balance or integrate a reaction trajectory.

For the reviewer, the useful result is an answer with an inspectable evidence chain: authorized source records, explicit model or tool inputs, returned calculations, limitations and the proposed next step. That is the practical distinction between an assistant that describes science and a workflow that executes scientific analysis. Source ownership, effective versions and decision rights remain part of the operating design.

A source-to-claim evaluation matrix

An original BioMedAna working framework. Adapt the decisions and evidence to the actual process, people and intended use.

The questions behind the headline

Follow the evidence

1. ISPE 2026 Annual Meeting & Expo - live agenda, accessed 21 October 2026 ↗
2. FDA - Data Integrity and Compliance With Drug CGMP, final, December 2018 ↗
3. NIST - AI RMF Playbook; voluntary guidance based on AI RMF 1.0 ↗
4. ISPE - Pharma 4.0 operating model: organizational, cultural, process and resource dimensions ↗

Original BioMedAna editorial analysis. The linked sources support the reported developments; examples, frameworks and product implications reflect our perspective.

Explore the underlying concepts

Explore the ISPE & Pharma 4.0 series ↗

• What is bioprocess data contextualization?
• BioMedAna Assist & Agents
• Governance and compliance
• Bioprocess data readiness checklist