Validating LLM judges for automated oversight of patient communication.

Xu Z, Zeng J, Zhou S, Zhang Z, Heintz T, Tonneau M, Yaghoubi A, Ye B, Goddla V, Lehmann L, Chen YH, Sharon E, Kozono DE, Revette A, Maues J, Brown T, Catalano P, Mak RH, Dligach D, Bitterman DS

Open source

DOI: 10.64898/2026.09.16.26363176
Published: 2026 Sep 17
Container: medRxiv : the preprint server for health sciences
Publisher: Not recorded
Open access: yes

Credibility signals

limited evidence Score 45/100 under policy 1.0.0. This is a metadata assessment, not a judgment of the paper's conclusions.

Show all credibility signals

cautionDOI registered: No matching Crossref record was present in this response. Source: Crossref; license: CC0 metadata
cautionDOI resolves: No matching Crossref record was present in this response. Source: Crossref; license: CC0 metadata
not scoredDirectory of Open Access Journals: No matching DOAJ record was present in this response. No allow-list match; this is not evidence of low credibility. Source: DOAJ; license: CC0
not scoredMEDLINE indexed: Not checked or no result supplied; no credibility inference made. Source: No authority result supplied; license: Unknown
not scoredOpenAlex core source: Not checked or no result supplied; no credibility inference made. Source: No authority result supplied; license: Unknown
not scoredKnown publisher allow-list: Not checked or no result supplied; no credibility inference made. Source: No authority result supplied; license: Unknown
not scoredROR affiliation: Not checked or no result supplied; no credibility inference made. Source: No authority result supplied; license: Unknown
not scoredRetraction Watch retraction: No retraction notice matched this DOI in the deployed snapshot. No matching event found; coverage may be incomplete. Source: Retraction Watch; license: CC BY 4.0
not scoredRetraction Watch expression of concern: No expression of concern notice matched this DOI in the deployed snapshot. No matching event found; coverage may be incomplete. Source: Retraction Watch; license: CC BY 4.0
not scoredRetraction Watch correction: No correction notice matched this DOI in the deployed snapshot. No matching event found; coverage may be incomplete. Source: Retraction Watch; license: CC BY 4.0
not scoredRetraction Watch reinstatement: No reinstatement notice matched this DOI in the deployed snapshot. No matching event found; coverage may be incomplete. Source: Retraction Watch; license: CC BY 4.0
supportingOpen access status: Normalized open-access status: open. Source: Normalized work metadata; license: Caller-provided; provenance license not supplied to scorer
not scoredPublication license: Not checked or no result supplied; no credibility inference made. Source: No authority result supplied; license: Unknown
not scoredPublication version: A publication version was supplied but is not scored. Source: Normalized work metadata; license: Caller-provided; provenance license not supplied to scorer
cautionMetadata completeness: 5 of 6 scored descriptive metadata groups are present; missing fields increase uncertainty. Source: Normalized work metadata; license: Caller-provided; provenance license not supplied to scorer

Cite this work

BibTeX

@article{allodium:10.64898/2026.09.16.26363176,
  title = {Validating LLM judges for automated oversight of patient communication.},
  author = {Xu Z and Zeng J and Zhou S and Zhang Z and Heintz T and Tonneau M and Yaghoubi A and Ye B and Goddla V and Lehmann L and Chen YH and Sharon E and Kozono DE and Revette A and Maues J and Brown T and Catalano P and Mak RH and Dligach D and Bitterman DS},
  year = {2026},
  journal = {medRxiv : the preprint server for health sciences},
  doi = {10.64898/2026.09.16.26363176},
  url = {https://doi.org/10.64898/2026.09.16.26363176}
}

RIS

TY  - JOUR
TI  - Validating LLM judges for automated oversight of patient communication.
AU  - Xu Z
AU  - Zeng J
AU  - Zhou S
AU  - Zhang Z
AU  - Heintz T
AU  - Tonneau M
AU  - Yaghoubi A
AU  - Ye B
AU  - Goddla V
AU  - Lehmann L
AU  - Chen YH
AU  - Sharon E
AU  - Kozono DE
AU  - Revette A
AU  - Maues J
AU  - Brown T
AU  - Catalano P
AU  - Mak RH
AU  - Dligach D
AU  - Bitterman DS
PY  - 2026
JO  - medRxiv : the preprint server for health sciences
DO  - 10.64898/2026.09.16.26363176
UR  - https://doi.org/10.64898/2026.09.16.26363176
ER  -

APA

Z, X., J, Z., S, Z., Z, Z., T, H., M, T., A, Y., B, Y., V, G., L, L., YH, C., E, S., DE, K., A, R., J, M., T, B., P, C., RH, M., D, D., & DS, B. (2026). Validating LLM judges for automated oversight of patient communication.. medRxiv : the preprint server for health sciences. https://doi.org/10.64898/2026.09.16.26363176

Source records

pubmed · retrieved 2026-09-25T20:44:04.160Z
europe-pmc · retrieved 2026-09-25T20:44:04.186Z