feat(api): add exponential backoff retry for 5xx errors

[HugoHSun]

Automatically retry API requests up to 3 times with exponential backoff (1s, 2s, 4s) when receiving 5xx server errors. This helps handle transient 502/503 errors during bulk page uploads. Note that although I wasn't able to reproduce the failed upload bug locally, this should still help in case it ever happens.

Ref: #1325

🚥 Resolves CX-2187

🧰 Changes

This PR adds automatic retry logic with exponential backoff to readmeAPIv2Fetch() to handle transient 5xx server errors that occur during bulk uploads.

Background

Users uploading large numbers of pages (66+) via rdme docs upload have reported intermittent failures with the error: Unexpected token '<', "<!DOCTYPE "... is not valid JSON.

This occurs when the ReadMe API returns a 502 Bad Gateway HTML error page instead of JSON during high load. The client-side JSON parser then fails on the HTML content.

Implementation

• Retry logic: Up to 3 retries for any 5xx response (502, 503, 504, 500, etc.)
• Exponential backoff: 1s → 2s → 4s delays between retries (10ms → 20ms → 40ms in tests)
• Behavior after retries exhausted:
  • 5xx responses → returns the response (caller handles via handleAPIRes)
  • Network errors → throws exception

Related

• Ref: Flaky rdme update docs: Unexpected token Error #1325
• Prior improvement: fix(fetch): handle non-JSON error responses better #1324 (better error messages)

🧬 QA & Testing

Automated Tests

The following scenarios are now tested in __tests__/lib/fetch.test.ts:

• ✅ Retries on 5xx errors and eventually succeeds
• ✅ Returns final 502 response after exhausting all retries
• ✅ Does NOT retry on 4xx errors (404, 400, etc.)
• ✅ Succeeds on first attempt without unnecessary delays

Manual Testing

• 4 processes uploading concurrently with 300 files each process