Mods and downloads

Design note (written before the code): a mod can add a mirror to the osu! source or an entire download source of VSRG maps, without ever touching the network itself. This document lists the choices to be validated; the final API is described in Downloads: downloads.register and the Download page in telechargement.md.

Principle

A script only declares data: URL templates, JSON paths, allowed hosts, a rate limit. All HTTP is done by crates/downloader (the same ureq/rustls client, the same limits as the built-in sources); nothing goes through the mods thread apart from the declaration, which is validated in Rust at registration (setup only, like gameplay.register). There is no search function written in script: it would require network access from scripts, which is refused.

defineMod({ id: "exemple.miroir", /* … */ permissions: ["network"], setup() {
  downloads.register({
    id: "mirror", name: "Exemple", kind: "mirror", target: "osu",
    hosts: ["mirror.example.org"],
    download: { url: "https://mirror.example.org/d/{id}" },
  });
}});

Two kinds, one single function:

• kind: "mirror", target: "osu": one more mirror for osu! beatmapsets (files by {id}, optional list). It joins the built-in mirrors in the order of the settings; a set of "osu! v2 beatmapset" responses is read as is (response.format: "osu").
• kind: "source": one more tab in Download, with search, a list of results (difficulty sets), and download of a set.

What a declaration contains

URL templates: typed variables ({query} text, {status}/{sort} via the declared tables, {offset}, {page}, {limit}, {cursor}, {keysMin}, {keysMax}, {starsMin}, {starsMax}, {bpmMin}, {bpmMax}, {lengthMin}, {lengthMax}, {genre}, {language}, and {id} for the file). Text is always percent-encoded, numbers are formatted by the host; there is no other variable. A parameter whose variable has no value is omitted; an optional group [ …] is written only if all of its variables have a value ({query}[ cs>={keysMin}]).

Response mapping: simple JSON paths (a.b[0].c, at most 8 levels, no filter or script) to id, title, artist, creator, status, bpm, playCount, favourites, cover, and to the difficulties (name, keys, stars, length, condition only: { path, equals }). Each value is typed and bounded by Rust (strings truncated, finite numbers, 1 to 18 keys); an unreadable result is ignored, never invented.

Security

1. network permission, declared in defineMod: without it, downloads.register fails. It is shown on the Mods page along with the hosts the mod will contact, and the player can revoke it mod by mod (networkDenied setting, like stageDenied): the mod's declarations disappear immediately.
2. HTTPS only, declared hosts only: every URL (template, URL read from a response, cover) is checked against the allowlist before being sent.
3. Redirects: followed manually, at most 3, each hop rechecked (https + allowlist); a hop to another host is refused; the token header is removed as soon as the host changes.
4. Private addresses refused: names that resolve to the local network, the loopback, or a link-local address are refused at resolution (a declared host cannot be used to probe the player's network).
5. Token: kept by the host (DPAPI, one file per source in <data>/download-tokens/, dedicated entropy), never given to the script, never in an event, an error message, or a log; sent only in the declared header, to the declared hosts, for this source. Covers do not carry it.
6. Limits (identical to the built-in sources): 8 MiB of JSON, 512 MiB per set, safe extraction (extract.rs: absolute paths, .., links, size recounted), received file that must start with PK, connection and response timeouts, 30 min per file.
7. Rate limit: sliding window per source; beyond it, the search responds "limited" with the delay, and a download waits (30 s at most) then fails.

Interface

• One tab per mod source, after Osu!/Quaver/Etterna, rendered by the page's generic component (the 8 themes share Download.svelte); each result is the card of an osu!-like set. Below the bar, a banner: "Source provided by <mod> · contacts <hosts>" and an icon to disable the source.
• Settings → Library → Downloads: ordered list of mirrors (built-in and from mods, with the providing mod and the hosts), move up/down, a toggle per mirror or mod source, token entry for sources that require one.
• Follows the mod lifecycle: disabling, failing, or uninstalling a mod removes its mirrors and tabs (the mirror list, ongoing cursors, and the active tab are cleanly restored).
• Added settings: networkDenied (mod ids), downloadMirrorOrder (keys), downloadDisabled (<modId>/<id> keys), all three specific to the machine and excluded from the settings export.

Choices to validate

1. Default allowed, removable: the declared network permission is active until the player removes it (like stage); the alternative would be "denied until it is granted". No request is sent before a search or a download.
2. Mod mirrors at the end of the order: they are added after the built-in mirrors and are therefore tried last in automatic mode, until the player moves them up.
3. Identifiers: a mod source result carries a text or number id; the interface and the queue use a numeric session identifier derived from (source, id), and the real id is kept on the Rust side and recorded (installed_pack.source = "mod:<modId>/<id>"). A mirror, on the other hand, requires a numeric id (the osu! beatmapset one).
4. No packs: a mod source lists sets (one zip archive = one folder), not Etterna-style packs with unfolded contents, no root folder kept.
5. Tokens: a single token per source (header declared among Authorization, X-API-Key, X-Auth-Token), no OAuth login and no cookie.
6. What cannot be described and remains refused: a search that requires several chained requests or a signature, HTML pages to parse, XML, POST, cookies, OAuth/login, a computed file URL. For these cases the source has to be added in crates/downloader.
7. Example: examples/mods/download-sources (not shipped, like the other examples); its test reads a frozen recording, never a real site.
8. Token of 8 to 4096 visible characters for all token-based sources (Quaver's minimum goes from 16 to 8: some API keys are short).
9. A forced mirror that has disappeared (mod disabled or uninstalled, mirror turned off) falls back to the automatic order instead of failing; an identifier that nobody ever had remains an error.
10. Settings excluded from export: networkDenied, downloadMirrorOrder, downloadDisabled (they name what is installed here).

Addendum: API bridges written by the mod (downloads.registerBridge)

Request: "the mod takes care of building the bridge for a particular API and of what needs to be displayed". Declarative sources remain as they are (JSON, GET, one request per page). For an API that does not fit this framework (POST with a form, XML or HTML response, a search → detail → download chain), a mod registers a bridge: pure functions, never any I/O. The principle "no network for scripts" still holds: it is the host that makes each request, the script only describes the next one.

downloads.registerBridge({
  id, name, hosts, auth?, rateLimit?,
  search(query, page, state)             -> Step,   // first page: page = 1
  onResponse(response, state)            -> Step,   // called with the response to the previous request
  action?(result, actionId, state)       -> Step,   // click on a result's button
})
// Step = { request, state? } | { results, nextPage?, state? } | { download, state? } | { error }

• request: { url, method: GET|POST, headers?, body?: string | {form} | {json}, expect }. The host checks https, host in the declared list, headers (allowlist of names; neither Host nor Cookie), body ≤ 64 KiB, then makes the request natively (redirects already bounded and re-checked), and calls onResponse({ status, headers, text }, state) back. Text ≤ 2 MiB; HTML and XML arrive as text (the script reads them with JSON.parse, strings, or regexes).
• At most 6 requests per search or per action, 60 s in total; one more step is a clear error.
• results: typed cards (id, title, artist, creator?, coverUrl?, tags?, size?, keyCount?, details?, actions), bounded and validated in Rust, rendered by the generic component (all 8 themes). coverUrl must be on a declared host and goes through the host's covers route.
• download: { url, method?, headers?, body?, filename?, format: zip|osz|qp }; the host downloads (size cap), extracts safely (zip-slip) and requires at least one chart file readable by ROX; installed like the other sources.
• Token: stored by the host (same TokenStore), injected into the declared header or in place of {token} in the URL, the body, or a header; the script never sees it (the host also erases the token from a response that would repeat it). No cookies, no OAuth.
• The functions run on the mods thread under the execution budget. An exception, an invalid output, or a budget overrun disables the bridge (message in the mod's log, source removed); the other sources keep working.

Choices to validate: 11. A bridge is only a source (tab), not a mirror; no filters/sorts for a bridge (the mod handles its own search). 12. Response too large: error (no silent truncation, a truncated JSON would be wrong). 13. 4xx/5xx statuses (except 401/403 with a token, 429) are handed to the script (status): an API may answer something useful with them. 14. xml/html: no native parseXml (no DOM, no added parser); JSON.parse is enough for JSON. 15. A click on an action reruns the script (action) at download time, on a download thread. 16. Accepted formats: zip, osz, qp (zip containers); rar/7z and standalone files remain refused. 17. A bridge is disabled (its source disappears, the mod's log says so) when one of its functions throws an exception, returns something JSON cannot write, or exceeds the execution budget; not when a returned step is invalid (unknown field, undeclared host, forbidden header): that is an error shown to the player, because the declaration is at fault, not the execution. 18. The token is replaced by [token] in any response handed to the script (a site that echoes it does not leak it); {token} is accepted in the URL, headers, and body only if auth is declared; never in the host name. 19. A coverUrl outside the declared hosts is ignored without error (the result remains); the filename of a download step is informational (the folder is named by the game). 20. Card rendering is generic (labels, size, details, buttons): the mod chooses the content, never the markup.