Defeating AWS WAF Challenges in Pure Node.js (No Browser)

#

Introduction

#

I have a small CLI called goodread that tracks my reading progress on Goodreads: ./goodread set pendulum 81 matches a currently-reading book by title fragment and POSTs a new percentage to one of Goodreads’ private web endpoints, authenticated with a cookie string copied out of my browser. It worked for months.

Then one day it started failing with:

error: HTTP 404 for https://www.goodreads.com/user_status.json: Page not found

A missing page? The route was right there in Goodreads’ own JavaScript bundles. And the weird part: reads worked perfectly. Listing my books, fetching my reading progress — all fine. Only writes died.

This post is the full story of peeling that onion: how a “404” turned out to be a lie told by an anti-bot system, and how I ended up writing a fake browser environment in Node.js capable of solving AWS WAF’s JavaScript challenges — no Chrome required. Everything below ships in a zero-dependency single-file library.

TLDR

#

• Goodreads’ write endpoints are gated by AWS WAF Bot Control. Without a valid aws-waf-token cookie, the backend answers with an app-rendered fake 404 ('Page not found', JSON content-type, X-Runtime header present) instead of a proper 401/403. Reads don’t require the token.
• Reimplementing AWS’s obfuscated proof-of-work VM from scratch is hopeless; the winning move is to execute their own challenge.js inside a vm.createContext fake browser and harvest the token when their code writes document.cookie.
• Hardest parts were not the crypto but boring plumbing details: cross-realm instanceof failures, duplicate object keys shadowing injected globals, worker-scoped navigator fields, and Blob API conformance checks.
• Final result: pure-Node token minting validated against production Goodreads (200 {"updatedAt":...}), plus a refresh loop we can ride indefinitely.

Act I: The Case of the Fake 404

#

The first instinct with any 404 is “the endpoint moved.” Let me walk through why that was wrong, because the diagnostic chain matters more than the conclusion.

The route exists

#

A logged-out POST gives a clean, structured response:

POST /user_status.json (no cookies)
→ 403 {"redirect":"/user/new"}

That is application-level JSON with a redirect instruction. Routes that don’t exist don’t know your sign-in URL.

Our session is fine… for reads

#

GET /, the shelf pages, and GET /home/load_reading_sessions all returned personalized data — my user id embedded in server-rendered React props, my actual reading sessions as JSON. Whatever authentication state we had, reads accepted it.

The refusal is deliberate

#

POST /user_status.json (our cookies)
→ 404, body: 'Page not found'
   Content-Type: application/json; charset=utf-8
   X-Runtime: 0.210983          ← Rails rendered this
   Set-Cookie: _session_id2=…   ← app processed it

Compare with a genuinely missing path (/zzz-no-such-page): that returns text/html with the full branded 404 layout. Our failure is a bare string with a JSON content-type — someone wrote code that renders exactly this when they don’t like you. It’s a soft-block disguised as a routing miss, presumably to make scrapers chase phantom routes.

Client fingerprint ruled out

#

Before blaming the server, eliminate the client. Same request via curl with HTTP/2, full Chrome header set (Sec-Fetch-*, Sec-CH-UA*, correct Accept chains), fresh CSRF token: identical 404. Python urllib, Node h1.1, curl — all identical. Not TLS fingerprinting, not header ordering, not protocol version.

Cookie bisection finds the real divide

#

Splitting the cookie jar told the real story:

Two revelations:

1. Authentication rides entirely on Amazon federation cookies (x-main, at-main, session-token, …). The Rails _session_id2 is just an anonymous container. Goodreads isn’t really “a website with a login form” anymore — it’s an Amazon property wearing a bookshelf costume.
2. Reads tolerate our (aging) federation tokens; writes re-validate harder and answer failure with the fake 404 rather than anything honest.

So the fix seemed obvious: refresh the cookie jar. Which worked! But it required a browser, and that bothered me…

Act II: The Puppeteer Detour

#

To understand what “fresh” means, I installed headless Chrome via puppeteer, injected my existing cookie jar into it, loaded Goodreads, and harvested everything the session produced: 18 cookies (up from 15), including a brand-new aws-waf-token, plus a fresh CSRF meta tag.

Then the decisive experiment — running the exact same fetch inside that browser, same origin, same credentials:

await page.evaluate(async () => {
  const r = await fetch('/user_status.json', { method: 'POST', … });
  return { status: r.status, text: await r.text() };
});
// → {"status":200,"text":"{\"updatedAt\":\"2026-08-23T16:41:56.373-07:00\",…}"}

Same jar contents, same endpoint: 200 inside Chrome, 404 from Node. The delta was the aws-waf-token cookie — issued after AWS WAF’s JavaScript challenge had run in a real browser. That’s the entire gate: reads pass without it, writes refuse without it, and the refusal is dressed up as a 404.

Great — problem understood, workaround exists (run headless Chrome once in a while). But ~150 MB of Chromium on a cron job to set one cookie felt absurd. The challenge script is just JavaScript. We have a JavaScript runtime. How hard could it be?

Act III: Going Browserless

#

The strategic choice

#

AWS WAF’s challenge is a ~700 KB obfuscated bundle containing a custom bytecode VM whose fingerprint checks rotate regularly. Reimplementing its math from scratch is a losing game — by the time you match this month’s build, next month’s differs.

But there’s another way to think about it: the challenge script is itself the most up-to-date documentation of what it needs. Don’t reverse the VM — give the VM a world to live in. Execute their script inside a fake browser and let them compute the token for us. Our job reduces to: fake enough DOM/browser surface that their script runs to completion without noticing.

One practical problem immediately: challenges are issued behaviorally, and after the browser solved one, our IP was in Goodreads’ good graces. Requests that got challenged all morning now returned clean 200s. You cannot develop a solver against a challenge you can’t reproduce.

Archaeology: mining Chrome’s cache

#

Conveniently, the earlier puppeteer run had already solved a challenge on this machine. Its disk profile should contain artifacts. A strings sweep over the cache files turned up gold:

$ strings Cache_Data/sqldb1 | grep awswaf
ea457862827c.18d38088.eu-north-1.token.awswaf.com
https://ea457862827c.18d38088.eu-north-1.token.awswaf.com/ea457862827c/ca63…/b63f…/challenge.js

Goodreads’ actual challenge.js deployment URL, recovered from a SQLite-ish cache file. Downloaded: 695 KB of var _0x436f28=['zxHWB3j0CW',...] beauty. Now we can iterate offline against the real thing.

The Design: A Browser in ~1000 Lines

#

The solver has three moving parts: fetch/parse the challenge response, execute the script inside a synthetic environment, harvest the cookie. Everything hinges on the middle part.

const ctx = vm.createContext(sandbox);
vm.runInContext(challengeJs, ctx, { filename: 'challenge.js' });
// then: fire lifecycle events, poll document.cookie until aws-waf-token appears

The sandbox object becomes the global scope of their script. Building it is an exercise in “what does minified fingerprinting code actually touch?” Here’s the layer cake, roughly in order of discovery pain:

Globals everyone forgets

#

Beyond the obvious window/document/navigator/screen/location/history, the script expected:

• document.currentScript.src — read during init to verify it’s loading from *.token.awswaf.com. Without it, init silently bailed.
• window.awsWafCookieDomainList — the protected site normally defines this inline before the script tag.
• Lifecycle events: nothing happens until you fire DOMContentLoaded then load through listener registries your addEventListener shims maintain.
• window.chrome with plausible app/runtime sub-objects — the classic headless-detection staple.
• performance.now(), crypto.getRandomValues, localStorage/sessionStorage maps, requestAnimationFrame driven by 16 ms timers.

Network interception

#

Their script speaks both fetch and XMLHttpRequest. Both shims route through one Node https.request wrapper that stamps Origin/Referer from the emulated page URL and feeds responses back as Response/XHR-shaped objects. Crucially, request bodies must support binary: ArrayBuffers, typed arrays, Blobs — more on that disaster shortly.

Workers in Workers

#

The challenge computes its proof-of-work inside Web Workers created from Blob URLs:

URL.createObjectURL(new Blob(['(', powFunction.toString(), ')()']))
new Worker(url)
worker.onmessage = e => collect(e.data);

Emulating this in Node was the fun part: a fake Blob stores its source text, createObjectURL registers source under a generated id, and Worker spins up a nested vm.createContext seeded with self, postMessage, timers, and crypto — then pipes messages between realms with setImmediate. Their PoW loop is time-sliced (for (end = Date.now()+4; Date.now() < end; …)), so it plays nicely with cooperative scheduling.

The cookie trap at the end

#

Everything converges on document.cookie. My shim keeps a real jar array, honors expires/max-age deletion semantics, and the driver polls for aws-waf-token= writes. When the script finally sets it, you’ve won.

The Iteration Saga

#

What follows is the honest log of every stall, because the bugs were better than the features.

Stall 1: silence after eval

#

Script evaluated cleanly, registered listeners for message and visibilitychange, then… nothing. No network calls, ever. Root cause was a stack of missing lifecycle pieces (events never fired, currentScript null). Lesson: instrument timers early — a healthy obfuscated script schedules work; total scheduling silence means an init branch bailed.

Stall 2: [object Object] as worker source

#

After fixing event plumbing, the script built a Blob and spawned a Worker whose source was literally the string [object Object]. Two separate bugs conspired:

1. A duplicate key in my sandbox literal — an early stub Blob: function(){} further down clobbered the real class (last-key-wins strikes again).
2. Even fixed, the probe worker came back with {"hc":8} — missing "pl" because my worker-realm navigator lacked platform. Their code validates the round-trip and degrades quietly otherwise.

Stall 3: the 20-byte lie

#

The probe passed; next came new Blob([someArrayBuffer], {type:'application/octet-stream'}) and then silence. My logs said size= 20 — suspiciously equal to the length of the string "[object ArrayBuffer]".

That’s exactly what it was: the buffer originated in the vm realm, so host-side p instanceof ArrayBuffer was false, my code fell through to String(p) coercion, and the real bytes never existed. Cross-realm objects break every identity check you take for granted. The fix everywhere: duck-type on byteLength/buffer instead of instanceof:

if (p && typeof p === 'object' && typeof p.byteLength === 'number') {
  // view or ArrayBuffer from ANY realm — copy the bytes, ask no questions
}

With binary plumbing honest again, the flow surged forward: cookie scans, then a proper multipart POST to mp_verify.

Stall 4: fighting a ghost

#

FormData still serialized empty. Probing the object at fetch-time revealed: own append = () => {}, prototype = {constructor} pointing at something named FormData… that wasn’t mine.

I burned real time on theories: anti-tamper decoys! capability-gated shims! defineProperty bypassing setter traps! I added accessor-pair traps to the prototype, Symbol.toStringTag brands, native-looking Function.prototype.toString output, even intercepted Object.create(FormData.prototype).

All reasonable defenses. All irrelevant — because the truth was in my own file:

Worker: SandboxWorker,
…
FormData: SandboxFormData,
FileReader: SandboxFileReader,
FormData: function () { this.append = () => {}; },   // ← the ghost, 400 lines later

An abandoned stub from an earlier edit, surviving as a duplicate key where last-one-wins made it clobber the real class. Every probe inside the vm saw the stub; every theory about “their decoy” was me fighting my own leftover line. And the non-writable freeze I’d added along the way locked the ghost in place, making it unfixable from inside the guest.

Audit duplicate keys first when injected globals misbehave. Cheaper than existential dread.

Anatomy of the Challenge (as observed)

#

Once unblocked end-to-end, the pipeline looks like this:

202 challenge page
 └─ <script src="…token.awswaf.com/<id>/<a>/<b>/challenge.js">
     └─ init: read currentScript.src, awsWafCookieDomainList, wait for load
         └─ spawn probe worker → expects {hardwareConcurrency, platform} back
             └─ Blob API conformance checks (ArrayBuffer parts, sizes)
                 └─ scan document.cookie for existing aws-waf-token
                     └─ PoW in workers → multipart POST …/mp_verify
                         └─ POST …/report (telemetry incl. failure stacks)
                             └─ document.cookie = "aws-waf-token=…"  🎉
                                 └─ refresh loop: POST …/telemetry
                                    {existing_token} → rotated token forever

Two things worth calling out. First, the telemetry endpoint happily accepts error reports — including your own stack traces from their failures — which is a gift for debugging (they even echo “Network response was not ok” back to themselves). Second, the refresh loop means one successful solve isn’t a point-in-time credential; the solver can keep rotating tokens indefinitely, riding their own keepalive machinery.

Validation Against Production

#

The end-to-end test (verify_e2e.js) does the whole dance with zero browsers:

1. Solve the saved challenge.js → mint token
2. Merge token into the goodread cookie jar
3. Read current progress (no-op target: write back the same percent)
4. Authenticated POST /user_status.json

token obtained: bbde34e1-80a1-4db8-b0a8-0adf5cc66082:DQoAknwKRx1ZAAAA:…
current percent of target book: 80 (percent)
WRITE PROBE -> 200 {"updatedAt":"2026-08-23T18:36:25.591-07:00",
                    "finalPosition":100,"currentPosition":80,…}

Then the mundane version: refresh-goodread.js merges a minted token into ~/.config/goodread/config.json, and the original CLI that started this whole mess works again:

$ ./goodread set pendulum 81
updated 'Stalking the Wild Pendulum…' to 81% -> 81 percent (final 100)

Lessons Learned

#

1. Status codes lie when bots are involved. A 404 from an anti-bot gate is cheaper for them than a truthful 403 that helps you tune your bypass. Trust content-type and timing fingerprints over status codes.
2. Cross-realm identity is a trap. Anything crossing a vm/iframe/worker boundary loses instanceof. Duck-type on structural properties (byteLength, _buf, method presence).
3. Duplicate object keys are silent overrides. Linters catch them in hand-written code, but generated/assembled literals slip through. When an injected global “doesn’t work,” grep for a second definition before theorizing about adversaries.
4. Instrument scheduling first. In obfuscated-code debugging, the fastest signal is often just logging every setTimeout/setInterval call: silence means an init branch bailed; a 10 ms heartbeat means you’re waiting on something specific.
5. Run their code, don’t reverse it. For rotating obfuscated challenges, a faithful-enough environment beats perfect understanding. The environment ages far better than any static decode of this month’s bytecode.

Notes on Scope & Ethics

#

This targets AWS WAF’s challenge mechanism for personal automation of my own account data on Goodreads (reading progress — the least exciting write imaginable). It deliberately does not touch CAPTCHA-mode (gokuProps), which exists precisely to demand a human. If you deploy something like this against services you don’t own, respect their terms, rate limits, and the letter of your account agreement; the techniques here are standard fare in QA automation and bot-management research, but applying them is your call.

Code Layout

#

The whole thing fits in a handful of stdlib-only files, no npm dependencies:

• waf-solver.js — the solver library + CLI (fetch/parse challenge, run script in the fake browser, harvest cookie)
• verify_e2e.js — production validation harness: solve → merge token → authenticated no-op write probe
• refresh-goodread.js — operational glue: mint + merge a fresh token into my CLI’s config

The README documents the environment layer by layer, including a full table of every stall and fix from the iteration log above.