Verify source URLs before an agent publishes

Separate the questions hidden inside verified

A source URL can be syntactically acceptable, belong to an approved publisher, return a page and support a claim. Those are four different observations. This workflow handles only a narrow string policy before any network access. It checks an explicit publisher allowlist and rejects ambiguous forms. The output is a normalized URL for later review. It is not a certificate of publisher credibility, current availability, copyright permission or the truth of an article.

State the conservative policy

The example permits HTTPS on three named documentation hosts, with no user information and no unexpected port. It rejects whitespace, control characters, non-ASCII characters, backslashes and every percent-encoded form. That final restriction deliberately excludes legitimate URLs that require encoding. The intended behavior is manual review, not an attempt to support every possible link. Exact host matching rejects a lookalike suffix such as a permitted name followed by another domain. Subdomains are not implicitly trusted.

Validate before using the parser’s result

Python’s URL parsing documentation explains that parsing does not validate a URL and that some control characters may be stripped. The code therefore rejects restricted raw characters before urlsplit. It then inspects the scheme, hostname, credentials, parsed port and canonical authority. A permitted explicit 443 port is removed during normalization. The path, query and fragment are retained, so an anchor can still identify a specific section. None of those steps opens a connection or follows a redirect.

Run the local fixture

Save verify_urls.py and run it using Python 3.12. The example normalizes a documentation link using the permitted default port. test_workflows.py checks allowed uppercase host spelling, rejected credentials, an unexpected port, a lookalike host, an IP address, a trailing-dot host, control characters and encoded forms. It also verifies that a fragment survives. These cases describe this policy’s exact boundary. They do not establish safe network access for every URL a browser or HTTP library can interpret.

Handle refusal without inventing a clean source

When a URL fails, preserve the originally supplied value in the protected review context and explain the policy reason. Do not strip suspicious material until the string appears acceptable, silently substitute a different publisher or invent a source date. If a legitimate encoded link is needed, an editor can review it and the engineering owner can define a tested extension. A one-off bypass should not become a hidden instruction that later links inherit broader authority.

Add network controls as a separate component

A future fetcher needs destination checks that account for DNS resolution, redirects, address changes, response size and timeouts. OWASP’s SSRF guidance addresses that larger boundary. This string-only demonstration makes no SSRF-protection claim because it does not implement or test those behaviors. Even an approved domain can redirect or host content that does not support the requested claim. Keep retrieval authorization and evidence assessment separate from the allowlist decision.

Keep publication with an accountable reviewer

Once a page has been retrieved through an authorized process, check its date, authoring organization, actual statement and limits. Preserve a citation another person can follow. A model may help find contradictions, but it should not treat a successful URL normalization as approval to publish. The useful result is a small deterministic gate that rejects obvious policy violations early while leaving the harder questions visible. That improves the research pipeline without hiding uncertainty behind the broad word verified.

Runnable Python fixture

"""Narrow source-URL syntax policy; performs no DNS, fetch or SSRF validation."""
from urllib.parse import urlsplit, urlunsplit

PUBLISHERS = frozenset({"www.python.org", "www.postgresql.org", "www.nist.gov"})

def verify_source_url(raw):
    if type(raw) is not str or not raw or len(raw) > 2048:
        raise ValueError("bounded URL required")
    # Reject before parsing: urlsplit can strip certain whitespace/control characters.
    if any(ord(char) < 33 or ord(char) > 126 for char in raw):
        raise ValueError("non-ASCII, whitespace or control character")
    if "\\" in raw or "%" in raw:
        raise ValueError("backslashes and percent encoding need manual review")
    try:
        parsed = urlsplit(raw)
        host = (parsed.hostname or "").lower()
        port = parsed.port
    except ValueError:
        raise ValueError("invalid authority") from None
    if parsed.scheme != "https" or host not in PUBLISHERS:
        raise ValueError("publisher not permitted")
    if parsed.username is not None or parsed.password is not None or port not in (None, 443):
        raise ValueError("credentials or unexpected port")
    if parsed.netloc.lower() not in {host, host + ":443"}:
        raise ValueError("noncanonical authority")
    return urlunsplit(("https", host, parsed.path or "/", parsed.query, parsed.fragment))

if __name__ == "__main__":
    print(verify_source_url("https://www.python.org:443/downloads/"))

Verification and references

The URL policy passed four fixture tests on 16 October 2026 using Python 3.12.10, including authority and control-character rejections. No URL was fetched. Run verify_urls.py for the example and test_workflows.py for the complete suite.

• Python 3.12 URL parsing documentation (living reference; accessed 2026-10-16)
• OWASP SSRF prevention guidance (living reference; accessed 2026-10-16)