Monitoring your own IP ranges: scheduled lookups through the API, the fields worth diffing between runs, and why a gap is not a zero

First, what is and is not on offer

Read the documentation end to end and you will find no monitoring, no alerts, no notifications and no webhooks, because there are none. A lookup is a lookup: it runs when something asks for it, its answer is cached with a lifetime per dataset, and a cached answer served past its lifetime is revalidated in the background. The footer line about re-checking on a schedule is the site's own housekeeping: the privacy policy says what gets searched decides what the site fetches, re-checks and archives next, and nothing in the docs describes a block being re-read on a holder's behalf or reported to anyone. The trend caption on a report says it as plainly as it can be said: points are days a lookup ran, and gaps are days nobody asked. So the schedule is yours to run, and the API is shaped to be run from one.

What it offers, as the docs state it. Every endpoint is HTTP GET, returning JSON except the streaming lookup, which returns server sent events. GET /api/v1/lookup?q= takes an IP, a CIDR prefix or an AS number in the same formats as the search bar; the path forms /api/v1/ip/{ip}, /api/v1/prefix/{net}/{len} and /api/v1/asn/{asn} return the same report and exist so you can build URLs without query string encoding. There is currently no key and no account. Requests are limited to 120 per minute per client IP; exceeding the limit returns HTTP 429 with rate limit headers, and the instruction is to back off until the window resets. The health check and the font and flag assets are exempt; everything else counts, including every page of a prefix list and every call to the history or archive endpoints. Anyone planning sustained high volume is asked to write to [email protected] first. The limit is per client IP, the address the request arrives from, which the whoami endpoint reports, so a fleet of hosts behind one NAT egress shares one budget, and the terms are unambiguous about the obvious workaround: rate limits are the only gate, and rotating IPs, keys or other tricks to evade them ends your right to use the service.

The rest of the terms, as far as they touch a job: search and the API are free today, and paid plans may come later, with notice and a free tier the site says it intends to keep; do not misrepresent the data as your own measurement, and do not bulk mirror the archive to resell it as it stands; there are no guarantees and no SLA, so verify before you act on a report. One more line is worth reading twice before the first run. Lookups you run are logged and become an input to the dataset, deciding what the site fetches, re-checks and archives next; the subject of a lookup appears in the public list of recent searches, as the normalised query, its kind and a time, with nothing about who searched; and a lookup can trigger the site's own checks on its subject. A daily job on your own space is exactly the traffic the site is built for. It is also not invisible.

Enumerate what you answer for

The job walks two lists. The first is what your network announces, and there is an endpoint for it: GET /api/v1/asn/{asn}/prefixes, paged with limit from 1 to 500, default 100, and offset; the page carries total, and each entry carries prefix, family, firstSeen, lastSeen and ongoing. Read it as an inventory rather than as history. The list covers a recent window of routing observation rather than the life of the announcement, an upstream default rather than a site setting, and reading a prefix's routing history gives the rough length of it from the prefix side: a block withdrawn long enough ago is not in this list at all, and one withdrawn recently appears with ongoing false until the window moves past it. total counts every entry, including those, so the number of blocks you announce now is the count of entries with ongoing true. On this endpoint key on prefix and ongoing alone; the two dates describe an observed window rather than the life of the announcement, so a diff of them can fire on a run where nothing changed.

Two more properties of this endpoint matter to a job. It documents only limit and offset; the offline and refresh parameters are documented for the lookup endpoints, not for this one. And it has one silence: the documented response carries asn, total, offset, limit and the prefixes list, and no sources array or meta, so nothing on the page says whether the dataset behind it answered. A total that falls to zero overnight is therefore suspect before it is a finding; the lookup report for the AS itself carries a sources array with the state of each dataset, and that is where the difference between withdrawn and unavailable can be read.

The second list is the space you hold that somebody else announces: blocks you have leased out, assignments to customers who run their own AS, anything on your registry record that your network does not originate. No endpoint lists blocks by holder, so this list is typed by hand from the registry record and kept next to the script, and leasing space out safely is the post that says why the lessor's side of it needs watching at all. The inventory is the first thing to diff, before any reputation field is read: an entry appearing in your announced list that you never configured, or one dropping to ongoing false when nothing was withdrawn, is a finding on its own, and two ASNs announcing the same prefix and hijacked or transferred are where to take it.

The request, and reading the response before trusting it

One lookup per block, and by the block, not by an address inside it: the docs put it as a bare address is not a /32, because looking up an address returns address level facts plus its covering block, and block level history comes from looking up the prefix itself. The path form /api/v1/prefix/198.51.100.0/24 is the one to build. Cold lookups are slow, once: the first lookup of a subject fans out to six sources and can take up to about 30 seconds, repeats inside a dataset's lifetime are answered from the cache in milliseconds, and a repeat after a dataset's lifetime has run out is still answered from the cache while that dataset is refreshed behind it, reading revalidating in sources. The first run of the job is the slow one, and a transfer timeout with room above the documented cold lookup time is the one number the script has to get right.

Read meta before storing anything. meta.partial is true whenever a dataset was still pending or had failed at response time; pendingDatasets counts the ones still being fetched, which finish in the background and land in the cache so that a repeat picks them up within seconds, and failedDatasets counts the ones whose source did not answer with nothing cached to fall back on, where repeating immediately does not help because the failure is remembered for a few minutes. meta.retryAfterMs is the wait after which the report is expected to be complete, taken across everything still missing; it is null on a complete report, and it lives in the body of a 200, because there is no Retry-After header on a lookup. Then read sources, one row per dataset with provider, dataset, ok, state, ageMs, fetchedAt and error. The state is one of fresh, revalidating, refreshed, stale, pending or error. Stale means the upstream failed and the last known good answer was served, and it does not set partial, so a report can be complete and still carry yesterday's answer for one dataset; error means nothing usable was cached, and the fields it would have filled are null, which the site renders as n/a and which is a missing answer rather than a zero. fullyCached says only that every dataset came from the cache, not what state each one is in, so the field to compare between runs is state per dataset, not the summary.

Two parameters and one endpoint to leave alone. refresh=1 forces an upstream refetch but is reserved for operators and is ignored without authorisation, so the job cannot force a re-read and should not try; offline=1 answers from cache only. The streaming endpoint delivers the same lookup as server sent events for a person who is waiting, and the docs' advice for batch work is the plain endpoint: fire the lookup, and if meta.partial is true, come back after meta.retryAfterMs. Store the whole document per run, dated, rather than a hand picked subset. The stated policy is that changes within v1 are additive, new fields may appear at any time and clients should ignore what they do not recognise, which cuts both ways: a whole document diff will fire on a field you never asked for, and a field you ignore today may be the one that matters next year. So keep everything, and diff a named subset.

What to store and what to diff between runs

By panel, with the field names the report uses. Registration: orgName, type, which is the registry's classification verbatim, status, which is an array of verbatim flags and needs sorting before it is compared, abuseEmail, the published abuse contact, which the report shows without saying whether it is the block's own or inherited from a parent, and parents, the enclosing blocks with cidr, name, handle, type and country. A changed abuse contact between runs is read in running an abuse desk, and a changed type in the whois status field. Routing: announced, which is true, false or null, and null means no routing source answered rather than withdrawn; prefix; origins, each an asn and a holder; rpki.status, valid, invalid, or unknown when no ROA covers the space, with the roas beside it; and moreSpecifics, the count of announcements inside the covering prefix. A status flipping to invalid is an RPKI-invalid prefix being dropped, and a new origin, or a new more specific row in originHistory, is the hijack or handover question above.

originHistory is one row per origin AS and exact prefix it announced, with scope, firstSeen, lastSeen and ongoing; scope says whether the prefix is the block itself, a covering block or a more specific inside it. Because a network that deaggregates produces several rows, reduce on asn before counting networks, and leave the covering rows aside, since they are somebody else's aggregate carrying the space for reachability rather than control. The diff lines that matter are a new row with ongoing true, a row for a more specific inside your block, and an existing row turning ongoing false.

intel samples the block on a prefix lookup, and intel.sampledFrom names the block the addresses were sampled from; a bare address lookup samples that one address, and an ASN lookup has no single block to sample and carries the space map described below instead, so the block level fields come from the prefix query. In intel.rollup: addressesChecked, the distinct addresses sampled; blacklisted, the addresses carrying that verdict from any feed; maxBlocklistCount, the most public blocklists carrying any one address, where null means no feed had a blocklist verdict at all, which is not zero; and blocklists, every list that flagged anything in the block, unioned, which the report prints as Blocklisted on and which is a different quantity from the count. Read intel.samplesTruncated with every one of them: true means the read hit its row cap and every count in the rollup is a floor rather than a total. intel.changes is the report's own list of verdict flips, each an ip, a provider, an at, a field, a before and an after, which the web report shows under Verdict changes we have recorded. It carries one date per line, at, the observation that noticed the flip, and the feed's value before and after it. It lists a feed's reading of an address moving between two observations: a verdict turning from yes to no or the reverse, and, in the same shape, a change in the risk label or in the network, organisation or country the feed attributes the address to. How many lists carry an address, or which ones, can move without any verdict flipping, and that drift shows up nowhere but in your own diff of the rollup. intel.flaggedShare is twelve months of pct, null for a month nobody looked. A climbing share is the whole range listing, read there rather than here.

tags.asn and tags.prefix are the confirmed labels on the origin network and on blocks covering or overlapping the subject. Key the diff on the set of tag_slug values across both arrays rather than on which array a slug sits in, because the report shows one chip per label and a label confirmed on both the network and a block inside it is listed once, under whichever claim is stronger; keep the cidr where a prefix tag carries one, and treat the notes, evidence and confidence on a tag as display text that can change without the tag changing. A block with no confirmed tag has simply not been labelled; the docs say absence of a tag is not a finding of innocence. insights[].code is the stable machine readable identifier for each finding, with a level of info, notice or warn, and the sorted set of codes is the cheapest line in the whole diff to alert on. On the ASN report, asn.space.blocks marks each shown block flagged, mild, clean or unsampled, with shown, announced, sampled and sampledTotal beside the list so the truncation is never silent; the blocks are a sample of the space rather than a census of every announced prefix, so diff them by their own prefix key and take the inventory from the prefixes endpoint, not from this map. Does your ASN affect your reputation reads that map in full. And sources itself: key on provider and dataset, compare ok, and compare state only for stale, error and pending, since fresh, revalidating and refreshed rotate with the age of the cached answer at the moment of the run; a dataset that has read stale for a week is a finding about the record, not about the block.

Cadence against the rate limit and the cache

The arithmetic of a run is one request per block plus the pages of the prefix list, against 120 a minute. A holder with forty blocks is nowhere near the limit and should still not burst, because the docs say what the limit does not: cold lookups are expensive for the service even when they are free for you, so spread the requests out and reuse the cached results your earlier requests created. A pause of a second or two between blocks costs nothing and keeps the job well inside the window. Two waits can appear and they are different things. A 429 comes with rate limit headers and the instruction to back off until the window resets; curl's --retry, when it is given, treats a 429 as transient and honours a Retry-After header when one is present, and because curl does not treat an HTTP error status as a failure by default, --fail is what stops the body of a 429 or a 5xx from being stored as the day's file. A partial report is a 200 with the wait in its body, so no flag on curl reads it; the script has to check meta.partial, sleep for meta.retryAfterMs, and repeat once.

How often is judgement, and the docs give no figure. Once a day is the cadence at which a holder can act on what the diff says, and more often earns nothing on a quiet block: a tighter loop mostly reads the same cached answer back. Around a cutover, a lease start or a delisting, the day the space changes hands is worth a run before and a run after, which the after transfer checklist asks for anyway. One consequence of the cache is worth knowing before the first alert: a cached answer past its fresh window is served immediately and refreshed behind it, marked revalidating in sources, so a change upstream can show in your file one run late. On the prefixes endpoint nothing marks that at all.

Where the run lives. Under cron, keep the command in a script file rather than on the crontab line, because a percent sign in a crontab command becomes a newline unless escaped, which bites a dated filename from date +%F before it bites anything else. Cron runs the job under /bin/sh with an environment it builds itself, so use absolute paths. Under systemd, OnCalendar=daily is midnight every day, Persistent=true makes a run missed while the machine was off fire at the next start, and RandomizedDelaySec= spreads the start. A minimal snapshot script, yours to harden:

#!/bin/sh
# One prefix lookup per block, one file per run, a pause between them.
# Crontab line: 17 4 * * * /usr/local/bin/subnet-snapshot
DIR=/var/lib/subnet-snapshots
mkdir -p "$DIR"
DAY=$(date -u +%F)
for BLOCK in 198.51.100.0/24 203.0.113.0/24; do
  NAME=$(printf '%s' "$BLOCK" | tr / _)
  curl -sSf --retry 3 --max-time 90 -o "$DIR/$NAME.$DAY.json" \
    "https://subnethistory.com/api/v1/prefix/$BLOCK"
  sleep 2
done

And the comparison, run against the previous file for the same block. The filter pulls the named subset above into one object, -S sorts object keys so that two files are comparable as text, and arrays whose order carries no meaning are sorted inside the filter:

FIELDS='{
  registration: .registration | {orgName, type, status: ((.status // []) | sort), abuseEmail, parents},
  routing: .routing | {announced, prefix, origins, rpki, moreSpecifics},
  origins: [.originHistory[]? | {asn, prefix, scope, ongoing}],
  rollup: .intel.rollup | {addressesChecked, blacklisted, maxBlocklistCount, blocklists: ((.blocklists // []) | sort)},
  truncated: .intel.samplesTruncated,
  tags: ([.tags.asn[]?.tag_slug, .tags.prefix[]?.tag_slug] | sort),
  insights: ([.insights[]?.code] | sort),
  sources: ([.sources[]? | {provider, dataset, ok,
    state: (if .state == "stale" or .state == "error" or .state == "pending" then .state else "served" end)}] | sort),
  partial: .meta.partial
}'
jq -S "$FIELDS" previous.json > old.txt
jq -S "$FIELDS" today.json > new.txt
diff old.txt new.txt

The last step is the notifier, and it is yours, because the site sends nothing. Cron already has one: it mails a job's output to MAILTO, or to the crontab's owner when that is unset, provided the host has a mailer, and without one the output goes to syslog instead. So a script that prints the diff and prints nothing when there is none has a working alert from the second run, once there is a previous file to compare. Whatever replaces it, the rule is the same: a difference between two runs is a prompt to open the report and read it, not a verdict.

What a difference means, and what a gap means

The archive semantics bound every line of the diff. The site keeps a dated point for a block only on the days somebody looked it up, so your schedule is the resolution of your record, and the days between two runs are unseen, not quiet. The report's own Changes list dates a flip at the observation that noticed it, one date per line; it does not date the day the feed changed its mind, and it is not the list's own listing or removal date, which the site does not hold. How blocklists work is the reference for what a feed's verdict is and is not. A daily lookup does not produce a daily reputation reading either: readings accumulate across lookups, the feeds are consulted on lifetimes the docs do not publish, and a lookup answered from the cache adds nothing new, which is why the trend on the report states its daysObserved and observations rather than drawing a line and hoping.

Some differences are not changes. A drop in blacklisted between runs can be sampling rather than recovery when samplesTruncated is set or the sampled set moved, and the docs' rule stands in both directions: unsampled is not clean, and absence of a result means not checked, not clean. A null is not a zero, in maxBlocklistCount, in announced, in a month of flaggedShare, in any field a dataset in error would have filled. A prefix total of zero can be an unavailable dataset. And a block whose first ever run already shows flags did not acquire them under your watch; why new ranges arrive blocklisted is that case. The archive endpoint, GET /api/v1/archive?q=, returns only the site's own records for a resource, change events written when a fetched record differed from its previous snapshot, up to 500 of them newest first, each with the provider, dataset, an eventType and a summary of what moved, plus the routing observations it has persisted with how many full BGP peers saw each. Nothing is fetched upstream to answer it, so a job that only polls the archive sees no new events for its block: the archive grows when lookups go upstream, and the lookups are the job.

What no diff can show: anything that happened between two lookups; why an origin changed, which is a phone call; which registry child holds an address inside the block, which the report does not list; or that a quiet address is clean. What it can show, run after run, is that the record you keep of your own space is at least as dense as anyone else's, which is the one advantage a holder has and the whole reason to run the job.

What the report shows for a block you watch, and what it cannot

For a CIDR query, the report shows how many addresses were checked and how many were flagged with the worst risk seen, a block map with one cell per sampled address and the unsampled remainder shown, Blocklisted on as the union of every list that flagged anything, each address on its own count of lists, and the samplesTruncated flag when the counts are floors; the origin history with first seen and last seen for every network that ever originated the space, and the line that no announcement of this exact prefix is recorded when the history holds nothing but covering routes; the RPKI status and the ROAs; and the Registration panel with the Organisation, the Type verbatim and the abuse contact the record resolves to. For an ASN query, the announced prefix list with its true total, and each shown block marked flagged, mild, clean or unsampled. All of it is available as JSON from the API, and the tools comparison is where this site sits among the ones that do offer a watch.

What it cannot do: monitor, alert, notify or email; see anything between two lookups; supply a list's own dated record of a listing or a removal; tell an abuse contact of your own from an inherited one; list the registry children under your block or a history of holder names; or re-read a subject on demand. The record it keeps is honest about its own gaps, and a job that keeps the same honesty, one dated file per run, a named subset diffed, and nothing read into the days between, is the monitoring that this site can truthfully be part of.