FDA's CDS Guidance Makes Explainability an Engineering Requirement

The FDA updated its Clinical Decision Support guidance on January 29, 2026. Most coverage will focus on what’s excluded from device regulation. The more important question is what isn’t.

Navigating the Updated FDA CDS Guidance

CDS software avoids device classification only if it meets four criteria. The first three are relatively mechanical: no image or signal analysis as input, medical information as the data source, and recommendations directed to a healthcare professional. Most teams building clinical AI can navigate those.

Criterion 4: The Human-in-the-Loop Mandate

Criterion 4 is different.

The software must enable the HCP to “independently review the basis for the recommendations” so they do not “rely primarily” on the output. The FDA interprets this to mean that the software itself—not a white paper, not a terms-of-service page—must surface input provenance, the algorithm’s rationale in plain language, the characteristics of validation data, and known gaps in patient-specific information.

Read that as an engineer, not a regulatory analyst.

The FDA is saying: if your system’s reasoning is not inspectable by the clinician at the point of decision — with identified inputs, described methodology, disclosed limitations, and patient-specific context for why this recommendation was generated for this patient — you are building a regulated medical device. That is not a labeling exercise. It is a system design constraint.

Why Clinical AI Explainability is an Engineering Requirement

This matters because most clinical AI treats explainability as a post-hoc artifact. A model card. A validation summary in documentation. The FDA’s guidance makes clear that is insufficient. Criterion 4 requires that the explanation be present in the software output at the moment the clinician is deciding, including the information used, what was missing, and how the recommendation was derived. If the clinician cannot trace the reasoning, the software fails.

Section V.C of the guidance catalogs how software functions fail Criterion 4. The pattern is consistent. In one example, a CDS tool provides a prioritized treatment list but does not identify which clinical studies matched the patient’s diagnosis or surface contraindications from drug labeling. In another, a mammography follow-up tool trained on 100,000 cases does not disclose input variables, dataset distribution, or the rationale for the recommendation. In both cases, the software is classified as a device—not because the recommendation was wrong, but because the HCP had no way to evaluate whether it was correct.

Nearly every failure is the same engineering gap: the clinician cannot see why the system said what it said.

Governed AI vs. Regulated Software as a Medical Device

At BiTE, we hold that AI is inherently probabilistic. What must be deterministic is the system around it — the schemas, the authority boundaries, the escalation paths, the evidence capture. We build governed AI with bounded action surfaces, qualification gates, scoped tool authority, and human-in-the-loop review enforced at decision points where risk warrants it.

Criterion 4 is the FDA reaching the same conclusion under regulatory direction: the control surface around clinical AI is what distinguishes a recommendation tool from a regulated device.

If your clinical AI cannot explain itself at the point of care, the FDA has made the consequence explicit.

Source: FDA, “Clinical Decision Support Software — Guidance for Industry and Food and Drug Administration Staff,” January 29, 2026.