Will my shared note show up on Google? How we keep note links out of search

A user wrote to us last week asking for a note to be taken down. Behind requests like that is a question we get in different forms: if I share a note, can a stranger find it by searching?

They should not be able to, and since 18 October we are much more confident of that. This is how note pages are kept out of search engines, and where the protection stops.

What a search engine could see on a note URL before 18 October

ShareNotes is a single-page app. The text of a note is not in the page the server sends; the browser loads a small HTML shell, then our JavaScript fetches the note from the API and shows it. Every note address, from /blue-fox-42 onwards, got the same shell.

That shell was the homepage. It said, in plain HTML:

<meta name="robots" content="index, follow">
<link rel="canonical" href="https://sharenotes.dev/">

Our app swapped the robots tag to noindex once the JavaScript ran. So the protection existed, but only for crawlers that run JavaScript, and only after the page had already said "index me". A crawler that reads HTML and stops saw every shared note as an indexable duplicate of our homepage. Made-up addresses such as /a/b/c did the same and returned 200 OK.

The note text was never in that HTML, so a non-JavaScript crawler had nothing private to index. But "it probably works" is not a good answer to "will my note show up on Google?", so we fixed it at the server.

Why robots.txt is the wrong tool

The obvious fix is to tell crawlers to stay away from note pages in robots.txt. It is also the wrong one, for a reason that surprises people. Google's documentation says a page blocked in robots.txt "can still appear in search results, for example if other pages link to it". And: "For the noindex rule to be effective, the page or resource must not be blocked by a robots.txt file."

In other words, robots.txt controls crawling, not indexing. If a public page links to a note and the note is blocked, a search engine can list the bare address without ever reading it, and it can never see the instruction not to. So our robots.txt deliberately lets crawlers in:

User-agent: *
Allow: /
Disallow: /admin

Notes also live at the top level of the site, next to pages we do want found, such as /markdown-notepad/. There is no /notes/ prefix to block even if we wanted to.

noindex in a header, not just a meta tag

A crawler reads response headers before it looks at any HTML, and long before any JavaScript runs. So the web server now marks note pages itself. The rule it uses is the same one the app uses to recognise a note address: a single path segment with no dot in it. For those, nginx adds a header and rewrites the shell before it leaves the server:

location @note {
    add_header X-Robots-Tag "noindex, nofollow" always;
    sub_filter '<meta name="robots" content="index, follow">'
               '<meta name="robots" content="noindex, nofollow">';
    sub_filter '<link rel="canonical" href="https://sharenotes.dev/">' '';
    try_files /index.html =404;
}

You can check any note yourself:

$ curl -sI https://sharenotes.dev/blue-fox-42 | grep -i robots
x-robots-tag: noindex, nofollow

Three details matter:

• The canonical link is removed rather than pointed at the note. A noindexed page should not claim to be the main copy of anything.
• nofollow is there because the links inside a note are someone else's content. We do not vouch for them.
• The plain-text version of a note, /api/v1/notes/<name>/raw, sends the same header. It is easy to forget that a second address for the same text needs the same protection.

The server does not check whether a note exists before adding the header. That would put a database query in front of every shared link someone opens. An address with no note behind it still gets noindex, and the app shows a "not found" page.

Real 404s for paths that can't be notes

Anything with more than one segment cannot be a note, so it now gets a real 404. The browser still receives the app, which renders our not-found page, so visitors see the same thing as before. Crawlers get an honest status code instead of another homepage copy. Landing pages and blog posts are real files, so they are served before any of these rules apply.

Routing rules like these break quietly: a later edit to the config can send note pages back to "index, follow" and nothing visible changes. So we keep a script that starts the real nginx config in a throwaway container and checks the status code and robots header for a list of addresses. It also checks the other direction: every page in our sitemap must stay indexable. It fails if any answer is wrong.

What noindex doesn't do

It is worth being precise, because the reassurance only goes so far.

• It does not make a note private. Anyone with the link can open it. noindex only stops search engines from listing the note page.
• It does not cover pages that link to your note. If you paste the link into a public forum, that forum page can be indexed, link included, and readers can click through.
• It is an instruction, not a lock. Major search engines follow it. A scraper that ignores robots rules is not stopped by it.

If a note holds something that should not travel, use the tools built for that: a PIN so the link alone is not enough, a short timer, or delete-after-first-read so the note is gone once your recipient has seen it. ShareNotes notes are not end-to-end encrypted, so keep long-lived secrets in a password manager.

And if a note needs to come down, send its link through the Suggestion box in the site header, as our terms describe.