A 201 that ignored every field: why text/json slips past express.json()

This is the note our API created, in full:

{"content":"seo-agent pastebinit probe","expiresIn":"1h","burnAfterRead":"true"}

That was the request body, stored as the note's text. The client asked for a one-hour note that is destroyed when first read. It got a note with no expiry and no burn, and the response was 201 Created with a link, exactly like a success.

The request came from our own weekly automated audit, which was checking whether pastebinit could use ShareNotes as a backend. As far as we can tell, no real user hit the bug: apart from the audit's two test notes, no stored note started with a JSON request body. It would have been easy to ship it to them, though, because nothing failed.

How express.json() decides what to parse

express.json() does not look at the body to decide whether it is JSON. It looks at the Content-Type header and compares it with its type option, which defaults to application/json. Parameters are ignored, so application/json; charset=utf-8 matches. text/json does not. When the type does not match, the middleware simply calls next() and leaves the body unread for whatever comes after it.

In our app, what came after it was this:

// Populates req.body as a string for text/plain bodies
const textBodyParser = express.text({ type: ['text/plain', 'text/*'] });

That parser exists for a good reason. The most common use of our API is piping a file or a command's output straight into a note:

curl --data-binary @error.log -H "Content-Type: text/plain" \
  https://sharenotes.dev/api/v1/notes

For that, the whole raw body is the note. So the handler has a simple rule: a string body is the content, and an object body is JSON with a content field and options.

Put the pieces together. text/json skips express.json(), matches text/*, arrives as a string, and becomes the note's content. Every option inside it is invisible. Each layer was correct on its own terms.

Who sends text/json?

The registered media type for JSON is application/json (RFC 8259), but text/json still turns up. pastebinit is one sender: in version 1.8.0, a pastebin configured with post_format = json is sent like this:

params = bytes(json.dumps(params), encoding='US-ASCII')
req.add_header('Content-Type', 'text/json')

pastebinit also sends every value as a string, and any field the user left blank as an empty string. So burnAfterRead arrives as "true", not true, and an unset expiry arrives as "". That matters for the fix.

Why a quiet 201 is worse than a 400

If our API had rejected the request, the sender would have seen an error and fixed the header. Instead it accepted the request and dropped the parts that limit exposure. Expiry and burn-after-read exist to make a note disappear. If we cannot read those flags, the honest failure is closed — no note — not open: a permanent note anyone with the link can read.

The general rule we took from it: when a request carries fields that restrict what you do, being unable to parse them is an error, not a default.

We did not go as far as rejecting text bodies that happen to look like JSON. Someone running curl --data-binary @package.json -H "Content-Type: text/plain" is sharing a JSON file, and that is a perfectly good note. The content type is the contract. The bug was that one content type meaning "this is JSON" was not honoured.

The fix and the tests

Widen what the JSON parser accepts:

app.use(express.json({
  limit: '11mb',
  type: ['application/json', 'text/json', 'application/*+json'],
}));

Then make the handler tolerate string-only clients without becoming loose about the flags that matter:

• Empty-string and null optional fields count as unset, so pastebinit's blank title does not become a title of "", and a blank expiry does not fail validation.
• burnAfterRead is on for true or "true", and off for anything else, including "false". A note should never be burned because of a parsing accident.
• Content types we do not understand still get a 400 rather than an empty note. That covers form-encoded bodies (curl's -d), octet-stream, and no content type at all.

The regression test sends exactly what pastebinit sends:

const res = await request(app)
  .post('/api/v1/notes')
  .set('Content-Type', 'text/json')
  .send(JSON.stringify({ content: 'from pastebinit', expiresIn: '1h',
                         burnAfterRead: 'true', title: '' }));

expect(res.status).toBe(201);
expect(res.body.expiresAt).toBeTruthy();
expect(res.body.burnAfterRead).toBe(true);

A second test pins down the case we deliberately kept: a JSON document sent as text/plain is still stored verbatim.

Making a JSON API work with regex-only clients

pastebinit cannot parse a JSON response either. It runs a regular expression over the response body and appends the captured part to the pastebin's base URL. Our API returns {"slug": "...", "url": "..."}, so the config captures the slug:

[pastebin]
basename = sharenotes.dev
https = True
post_page = api/v1/notes
post_format = json
paste_regexp = "slug"\s*:\s*"([^"]+)"

[format]
content = content
title = title
expiry = expiresIn

[defaults]
expiry = 24h

One trap, if you write a config like this for your own service. pastebinit reads https as a string, so https = False still means HTTPS, because any non-empty string is true. To use plain HTTP, leave the line out.

The full setup, and why pastebinit needed a new backend in the first place, is in Ubuntu Pastebin is gone and pastebinit is broken.