Acquire from any source, safely.
Coming soon
Submit a link or call the API. {{PRODUCT_NAME}} resolves the source, maps every file and acquires what your policy selects, verified and recorded.
Sources
- Direct linkshttps://archive.example.org/…
- Web listingsIndex of /pub/2026/
- Onion services….onion
- Peer-to-peer swarmsmagnet:?xt=urn:btih:…
Published, newest first
- attachments/samples.zip1.4 GiBChecksum from the source verified
- attachments/bulletins/bulletin-2026-07.pdf16.9 MiBChecksum computed on arrival
- captures/listing-0001.html428 KiBChecksum computed on arrival
- attachments/indicators-0417.json2.3 MiBVerified by the transfer protocol
- screens/page-0001.png717 KiBChecksum computed on arrival
- attachments/bulletins/bulletin-2026-06.pdf14.2 MiBChecksum from the source verified
- shared/case-0417/notes.txt6.2 KiBChecksum from the source verified
Everything a link can lead to.
One generic pipeline resolves every source, known or not. Standards, reviewed recipes and versioned connectors widen it without new code in the core.
Only sources you are authorized to collect from. Missing access becomes a request for action, never a workaround.
Direct links
Reached throughVPN tunnelsTor pool
Direct, shared and temporary links, redirects and the intermediary pages between them.
- https://
- http://
- 302 Found
- ?expires=…
Web listings
Reached throughVPN tunnelsTor pool
Server indexes, catalogues, tables, detail pages, galleries, feeds and paginated JSON listings.
- Index of /
- ?page=2
- <enclosure>
- next_cursor
File servers
Reached throughVPN tunnels
FTP and FTPS, SFTP with pinned host keys, and WebDAV collections, folder by folder.
- ftp://
- ftps://
- sftp://
- PROPFIND
Cloud storage
Reached throughVPN tunnels
Configured storage remotes and shared folders, listed through their own interfaces.
- remote:bucket/
- /shared/
- next_page_token
File hosts
Reached throughVPN tunnels
Hosting services with link formats of their own, resolved by dedicated adapters and isolated engines.
- /file/…
- /folder/…
- #key
Dynamic pages
Reached throughVPN tunnelsTor pool
Links that only a script builds are found by a sandboxed browser, within strict budgets.
- <script>
- fetch()
- subresources
Onion services
Reached throughTor pool
Onion addresses go only to the isolated Tor pool and never touch ordinary DNS.
- .onion
- client auth
Peer-to-peer swarms
Reached throughPeer sessions
Magnet links and torrent files run in tunnelled sessions of their own, never over Tor, with no seeding afterwards.
- magnet:?xt=
- .torrent
- info-hash
Archives and sets
Reached throughAny network
Archives are listed member by member, and the parts of a multi-part set are chosen as one.
- .zip
- .tar.gz
- .7z
- .rar
- .iso
- .part1
Authorized access
Reached throughVPN tunnelsTor pool
Passwords, tokens, keys and onion client keys are sent once, kept as references and never shown again.
- username_password
- token
- ssh_key
- onion_client_auth
One link becomes a complete inventory.
Resolution gathers evidence before it believes anything. A size stays unknown until proven, and a listing is complete only when every branch closes.
- archive.example.org/collections/0417/collection
- attachments/83 files1.9 GiB
- bulletins/79 files282 MiB
- indicators-0417.jsondata2.3 MiB
- samples-part2.7zarchivesize unknown286 MiB
- samples.ziparchive1.4 GiB
- readme.txtarchive member
- docs/overview.pdfarchive member
- 10 more members
- unknown-0042.binunknown type3.1 MiB
- captures/148 files46.1 MiB
- index.htmlintermediate page48.0 KiB
- README.txttext2.1 KiB
- screens/64 files41.1 MiB
- video/2 files1.5 GiB
- recording-02.mp4video1.1 GiB
- walkthrough.mp4video412 MiB
- ValidateScheme, port and host class are checked without contacting the source.
- Choose the networkAn onion name selects the Tor pool, any other name a VPN tunnel.
- Gather evidenceHeaders, signatures, listings and range reads, cheapest first.
- ClassifyFile, folder, collection or intermediate page, each with its evidence.
- EnumerateFolders page by page, archives member by member, cycles cut.
- Close the listingComplete within scope only when no branch is left open.
{"location_id": "loc_01JAH8K2","name": "samples.zip","depth": 1,"has_children": false,"listing_complete": true,"resource": {"resource_id": "res_01JAH8K2","kind": "file","kind_status": "confirmed","discovery_state": "resolved","size": {"bytes": "1481763717","status": "validated","basis": "representation"},"media": {"category": "archive","identification_status": "confirmed","detected_media_type": "application/zip"}}}New sites, without new code
Standard protocols need only configuration. Known page layouts get declarative recipes. Novel protocols get a versioned connector, never a core edit.
- StandardsWeb listings, FTP, SFTP, WebDAV and storage remotes, understood out of the box.
- RecipesFetch, select, extract, emit. No scripts, no code, only operations the platform allows.
- ConnectorsVersioned packages that run out of process and pass a conformance kit first.
How a recipe earns its place
- Proposed
- Tested on fixtures
- Canary on a share of the work
- Promoted on evidence
- Withdrawn on drift
A model may propose a recipe's steps. People review it, and only evidence promotes it.
{"recipe_key": "archive-listing.paged","version": 3,"language_version": "1.0","scope": {"hosts": ["archive.example.org"],"path_patterns": ["/collections/*"]},"steps": [{"op": "fetch","target": "input_url","accept": "html"},{"op": "select","language": "css","expression": "table.listing a.file","as": "links"},{"op": "extract","from": "links","attribute": "href","as": "hrefs"},{"op": "emit","kind": "unknown","native_id_from": "hrefs","relation": "contains"}],"expected_evidence": {"exhaustiveness": "provider_declared"}}Fictional values, field names exactly as the API defines them
Nothing downloads before the order is proven.
Rules decide what is allowed, left out or quarantined. Strict selection waits for a closed listing and known sizes, then ranks every file by size.
Policy rules, highest priority first
- 1exclude-videoPlayable video is left out.
- 2inspect-unknownUnknown types are inspected; inconclusive means quarantine.
- 3opaque_encrypted_when_acquiredEncrypted archives stay opaque until someone provides the password.
- 4default_actionEverything else is allowed.
Recorded with the selection
strict_order_closed_inventory
Every file of the complete inventory was ordered by size before downloading.
Selection modes
- 1Every file, largest first
- 2The largest file
- 3The largest file of each folder
- 4Files within a size range
Strict mode waits for a closed listing and known sizes. Progressive mode starts early, and says so.
{"candidate_id": "rep_0008D","display_name": "walkthrough.mp4","candidate_kind": "representation","selection_revision": 2,"action": "exclude","rule_id": "exclude-video","reason_codes": ["video_excluded"],"evidence": [{"mechanism": "format_header","attribute": "kind","status": "confirmed","summary": "video/mp4"}]}Every request leaves through a tunnel.
Probes, listings and transfers alike use authorized egress. When a tunnel drops, requests stop: there is no direct route to fall back on.
Every request crosses the policy gate, then leaves through the lane its source requires.
Onion traffic takes layered Tor circuits, in an isolated pool of its own.
Swarms are joined from peer sessions of their own. Pieces arrive, and nothing is seeded afterwards.
Tunnel down. New requests stop, transfers checkpoint and nothing leaves directly.
Tunnel back. Transfers resume from their checkpoints.
- Private, loopback and metadata addresses are refused at connection time.
- Onion names never touch ordinary DNS.
- Peer sessions stop sharing as soon as the requested files are complete.
- A revoked binding closes live connections, and the gateway acknowledges it.
- When a stop cannot be confirmed, its capacity stays held until it is.
- Partials live in object storage, so another unit can resume them.
Verified, recorded, never executed.
An independent verifier hashes every staged file. Bytes are stored by content, published once and served only as attachments.
A mismatch is quarantined and acquired again, never published.
Two locations, one blob
Identical bytes are stored once by their SHA-256. Every location that pointed to them is kept.
Kept as evidence, never run
- Nothing acquired is opened, run, installed or mounted.
- Analyzers run without network, as their own user, with time and memory limits.
- Archive paths are checked before a single member is extracted.
- Files leave only as attachments, through short-lived links on a separate origin.
{"file_id": "file_0008","logical_path": "attachments/samples.zip","stored_name": "21d80013bfc9266b","size_bytes": "1481763717","sha256": "21d80013bfc9266bc453a3db98d1f3d14a2be4483d5020d74ec194a763336753","integrity_level": "origin_checksum_verified","origin_checksum": {"algorithm": "sha256","value": "21d80013bfc9266bc453a3db98d1f3d14a2be4483d5020d74ec194a763336753"},"source_resource_id": "res_01JAH8K2","published_at": "2026-10-05T08:31:00Z"}Fictional values, field names exactly as the API defines them
Built to be called, ready for the parser.
Any project submits jobs through the API and follows them by events. The parser takes each published file with its checksum and provenance.
- Your project
- Downloader
- Parser
- Your projectDownloaderSubmits a jobA job with an idempotency key: a retry never creates a second job.
The job your project sends {"sources": [{"url": "https://archive.example.org/collections/0417/"},{"url": "https://share.example.net/s/K7q2xWm9"}],"policy_profile": "evidence-default","destination_id": "evidence-store","client_reference": "Archive capture, case 0417"} - DownloaderYour projectAccepts it once storedAnswered only once the job and its policy are stored.
What comes back {"job_id": "job_01JAH7Q2M4K8V3","policy_revision": "prv_4","accepted_at": "2026-10-05T08:12:04Z","idempotent_replay": false} - DownloaderParserAnnounces a published fileSigned webhooks, or events you poll or stream from a cursor.
The event the parser receives {"event_id": "0f6e2a90-41d7-4c0b-9e5a-000000000010","event_type": "file.published","schema_version": "1.0","occurred_at": "2026-10-05T08:31:12Z","job_id": "job_01JAH7Q2M4K8V3","aggregate_type": "file","aggregate_id": "file_0008","aggregate_version": 3,"stream_seq": "10","data": {"file_id": "file_0008","logical_path": "attachments/samples.zip","size_bytes": "1481763717","verification_level": "origin_checksum_verified","selection_revision": 2}} - ParserDownloaderReads the file recordThe parser checks that the file is committed and reads its SHA-256.
The file record the parser checks {"file_id": "file_0008","job_id": "job_01JAH7Q2M4K8V3","logical_path": "attachments/samples.zip","original_name": "samples.zip","size_bytes": "1481763717","sha256": "21d80013bfc9266bc453a3db98d1f3d14a2be4483d5020d74ec194a763336753","integrity_level": "origin_checksum_verified","state": "committed","detected_media_type": "application/zip"} - ParserDownloaderFetches it by a short-lived linkA short-lived link; the parser hashes the bytes again before parsing.
Inventory, manifest and decisions also export as NDJSON.
Fictional values, field names exactly as the API defines them
See it on your own sources.
Tell us what you need. We will walk you through a live capture, from post to sealed record.
Prefer email? Write to {{CONTACT_EMAIL}}.