How often is the page checked?

Once an hour, taking at most the latest 20 items.

Can I use it without signing in?

No. It is a signed-in feature because fetching is expensive.

How do I stop this being created for my site?

Disallow the URL in your robots.txt. No request to us is needed; it stops automatically.

Following a site that has no RSS feed

Sites that publish no feed can still be followed: updates are detected from the page where the posts are listed.

Plenty of sites publish no feed

Company news pages, government sites and personal diaries often have no RSS at all. Some used to and dropped it during a redesign.

This service can read headings off such a page and assemble them into a feed. It is available once you are signed in.

How to use it

When adding a feed, enter the URL of the page where posts or updates are listed. If no feed is found, the option to read headings off that page is offered.

It does not suit a page holding a single article, or a front page whose contents simply rotate. A page with dated titles running down it works best.

Fetching rules

Because this is expensive to fetch, it is treated differently from an ordinary feed.

The page is checked once an hour
At most the latest 20 items are taken from a page
Article bodies are not fetched — you get titles and links
It cannot be created for sites that disallow it in robots.txt. Both our User-Agent and * are evaluated, and the stricter one wins

How it differs from a real feed

Because it works from the site’s HTML, a change to the page layout changes what gets picked up. Missed updates and the occasional non-article are both possible.

Per-item IDs and exact publication times are not available the way a feed provides them, so ordering can come out differently from what you expect.

If the site brings its feed back, switching to it is the more reliable option.

If you run a site and want this stopped

Adding a Disallow for the URL in your robots.txt stops it automatically — nothing needs to be requested from us.

For anything else, such as deleting feeds already generated or removing public pages, there is a form linked from the crawler page. Requests are processed after verifying domain ownership automatically.

FAQ

Are article bodies fetched?: No. Only titles and links.
How often is the page checked?: Once an hour, taking at most the latest 20 items.
Can I use it without signing in?: No. It is a signed-in feature because fetching is expensive.
How do I stop this being created for my site?: Disallow the URL in your robots.txt. No request to us is needed; it stops automatically.