Case Study · Production

MadridFlow

A bilingual editorial platform for Real Madrid coverage. Built and operated in production as a solo engineering project.

Role Solo engineer — architecture, backend, frontend, DevOps, SEO
Stack Node.js · React 19 · MongoDB · Nginx · Cloudflare · Anthropic · OpenAI
Status ● Live — madridflow.com ↗
50+ API endpoints 8 sitemaps · hreflang ES/EN Dual AI provider ~13K LOC backend 9 MongoDB schemas 6 SSR controllers

Summary

MadridFlow is a bilingual editorial platform covering Real Madrid — built and operated in production as a solo engineering project. The system spans a React 19 SPA, a Node/Express backend with 50+ routes, MongoDB Atlas, Nginx, Cloudflare, and a custom AI pipeline that generates editorial titles and captions in both Spanish and English.

It is not a demo: it serves real content, handles real traffic, and makes real tradeoffs.

The problem

Sports editorial content is fast-moving, media-heavy, and internationally distributed. A tweet-based editorial workflow creates three distinct engineering challenges:

  • Media delivery. Twitter's CDN blocks direct embedding when used outside their widget. Loading the official Twitter widget destroys Core Web Vitals — INP, CLS, and LCP all regress measurably.
  • Discoverability. Social media content is opaque to search engines. Without structured URLs, SSR, and sitemaps, none of the editorial work is indexable.
  • Editorial scale. Writing bilingual titles, meta descriptions, captions, and structured summaries for every piece of content is not sustainable manually.

The solution

Three focused engineering investments:

  1. A streaming video proxy that fetches from video.twimg.com server-side, serves partial content with 206 Range headers, and exposes the stream through a rate-limited, SSRF-hardened endpoint — giving the frontend native <video> performance with zero external JS.
  2. A 15-route Nginx + Express SSR architecture where every content type (stories, tags, topics, players, matches, authors) gets its own Express controller, full HTML response, structured JSON-LD, hreflang, and a place in one of 8 sitemaps.
  3. A dual-provider AI pipeline using Anthropic Claude (primary) with OpenAI Whisper/GPT-4o-mini (for ASR and translation fallback) — batched, with cost tracking and robust JSON recovery from malformed model outputs.

Architecture

The system runs as a Docker Compose stack on a Linux VPS, behind Cloudflare's CDN. Nginx is the single entry point, routing requests to either the Express backend (SSR + API) or the React SPA (catch-all). 

Browser
  └─ Cloudflare  (CDN · origin certs · staging rules)
       └─ Nginx  (Alpine · reverse proxy)
            ├─ /es/story/:slug  ──────────────────┐
            ├─ /es/tag/:slug  ────────────────────┤
            ├─ /es/tema/:slug  ───────────────────┤  Express SSR
            ├─ /es/jugador/:slug  ────────────────┤  (backend:5000)
            ├─ /es/autor/:slug  ──────────────────┤
            ├─ /sitemap*.xml  ────────────────────┘
            ├─ /api/*  ───────────────────────────  Express JSON API
            └─ /  ────────────────────────────────  React SPA (catch-all)

Express  (Node.js 18)
  ├─ server.js (535 lines)     — rate limiting · Helmet CSP · health
  ├─ videoProxyRoutes.js       — streaming proxy · SSRF hardening
  ├─ seoRoutes.js (459 lines)  — 8 sitemaps · robots.txt
  ├─ megaPromptService.js      — AI pipeline (Claude + OpenAI)
  └─ tweetCache.js (170 lines) — LRU cache · deduplication

MongoDB Atlas
  └─ 9 schemas: Day (tweets[]) · Topic · Player · Match · Author · …

React 19 SPA
  └─ SmartTweet.jsx (47KB) · Story.jsx (48KB) · HomePage.jsx (30KB)
  └─ GA4 lazy-loader  (impact-zero · post-FCP)

Engineering decision

SSR routes are registered in Express before express.static(). Getting this wrong silently serves the SPA shell to crawlers — no error, just invisible to Google. The ordering is load-bearing.

Key features

LRU Cache with Request Deduplication

server/src/services/tweetCache.js · 170 lines

A 2,000-entry in-memory LRU cache with three properties that go beyond a simple Map:

  • Request deduplication. If two requests for the same tweet arrive simultaneously, a single upstream fetch is made and the result is shared via a shared Promise.
  • Selective caching. A shouldCache predicate prevents storing transient errors (5xx, network failures) while caching deterministic results (404, 403).
  • Mutation safety. structuredClone on both read and write prevents callers from corrupting cached state. 
withCacheAndDedupe(key, fetchFn, { ttlMs, shouldCache })
  ├─ Check cache     → return structuredClone(entry.value)
  ├─ Check in-flight → join existing Promise, return clone
  └─ Fetch upstream  → cache result, resolve all waiters

shouldCache: (result) => {
  if (result.ok === true) return true;
  // Don't cache transient failures (network errors, 5xx)
  const isTransient = result.reason === 'error' ||
                      (result._status && result._status >= 500);
  return !isTransient;
}

Streaming Video Proxy

server/src/routes/videoProxyRoutes.js · 960 lines

  • SSRF hardening. URL allowlist validates HTTPS protocol + video.twimg.com domain before any upstream request is made.
  • 206 Partial Content. Range header forwarded to upstream, partial response relayed — enables seek/scrub without full re-download.
  • Smart variant selection. pickBestMp4Variant() filters by codec, sorts by bitrate — always serves the best quality the connection supports.
  • Token computation. ((id / 1e15) * π).toString(36) — reverse-engineered Twitter Syndication API authentication token.
  • Rate limiting. Keyed on {ip}-{sha256(url).slice(0,16)} — 600 req/min per IP+URL pair, burst-friendly for seeking.

Dual-Provider AI Pipeline

server/src/services/megaPromptService.js · 714 lines

claude-sonnet-4 as primary model. OpenAI GPT-4o-mini/Whisper-1 for ASR and translation fallback.

  • Batched generation. Tweets split into groups of 5, processed with Promise.all(), with partial-success handling — failed batches are reported, not silently dropped.
  • Prompt caching. Anthropic's cache control headers reduce redundant token costs on repeated editorial runs. Cache-read rate: $0.30/1M vs $3.00/1M for input.
  • Robust JSON recovery. A 7-step parser handles markdown fences, trailing commas, smart quotes, and unterminated strings — all real failure modes from production runs.
  • Token budgeting. Math.min(2000 + (tweetCount * 2000), 16000) — scales with content, caps at the model's context limit. 
// Cost tracking per request
Input tokens:       $3.00 / 1M
Output tokens:     $15.00 / 1M
Cache-write:        $3.75 / 1M
Cache-read:         $0.30 / 1M  ← 10× cheaper on re-runs

// Batched generation
generateEditionWithBatching(tweets, context, batchSize = 5)
  ├─ Split into batches
  ├─ Process all: Promise.all(batches.map(processBatch))
  ├─ Aggregate: successful + failed + missingIds
  └─ Return: { success, partial, data, usage, batch_summary }

Multi-Content Express SSR

6 SSR controllers each render a complete HTML document (not an SPA shell) for their content type. Every controller follows the same contract:

  • Query MongoDB with .lean() for raw POJO performance.
  • Emit noindex when content doesn't meet quality thresholds — enforced server-side, not via robots.txt patterns.
  • Inject structured JSON-LD: Article, BreadcrumbList, Person, ItemList.
  • Set Cache-Control: public, s-maxage=300, stale-while-revalidate=60 — Cloudflare edge serves returning crawlers without hitting Express.
/es/story/:slug storyController.js Article + BreadcrumbList
/es/tag/:slug tagController.js BreadcrumbList · noindex < 5 stories
/es/tema/:slug topicController.js BreadcrumbList · editorial cluster
/es/jugador/:slug playerController.js Person + BreadcrumbList
/es/partido/:slug matchController.js Event + BreadcrumbList
/es/autor/:slug authorController.js Person + ItemList

Performance & scalability

  • GA4 deferred. Loads on first user interaction (pointerdown / keydown / scroll) — does not touch LCP or FCP. client/src/analytics/ga4.js
  • Zero Twitter widget JS. No tweet widget on any SSR page. INP/CLS/LCP remain clean. The video proxy replaces the widget entirely.
  • Edge caching. Cache-Control: public, s-maxage=300, stale-while-revalidate=60 on all sitemap responses — Cloudflare serves crawlers without hitting the origin.
  • MongoDB connection pool. max 10, min 2 connections, configured in docker-compose.yml — no per-request connection overhead.
  • Dynamic changefreq. Sitemap entries get "daily" for stories published in the last 7 days, "weekly" for older — correct crawl-budget signals.

SEO & discovery

The SEO architecture treats discoverability as an engineering system, not a configuration task.

  • 8 sitemaps in a sitemap index: stories-es, stories-en, tags, topics, players, matches, authors, pages. server/src/routes/seoRoutes.js · 459 lines
  • Hreflang. Every story sitemap entry includes <xhtml:link rel="alternate"> for ES, EN, and x-default.
  • Programmatic noindex. Tag pages with fewer than 5 stories, unpublished authors, draft matches — all enforced server-side. Cleaner and more reliable than robots.txt patterns.
  • Googlebot log monitoring. nginx/default.conf includes a map directive detecting Googlebot/Bingbot/Applebot by User-Agent, logging them separately — crawl frequency monitoring without third-party tools.
  • robots.txt generated by Express. Staging subdomain receives Disallow: /; production receives correct allow/disallow rules + sitemap reference. 
<sitemapindex>
  <sitemap>sitemap-stories-es.xml</sitemap>   ← story pages (ES)
  <sitemap>sitemap-stories-en.xml</sitemap>   ← story pages (EN)
  <sitemap>sitemap-tags.xml</sitemap>          ← tag hubs (≥5 stories)
  <sitemap>sitemap-topics.xml</sitemap>        ← editorial clusters
  <sitemap>sitemap-players.xml</sitemap>       ← player profiles
  <sitemap>sitemap-matches.xml</sitemap>       ← match pages
  <sitemap>sitemap-authors.xml</sitemap>       ← author profiles
  <sitemap>sitemap-pages.xml</sitemap>         ← static pages
</sitemapindex>

Security & best practices

  • No secrets in code. All credentials via environment variables (ADMIN_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY).
  • XSS prevention. escHtml() utility applied to all user-facing content in SSR templates — no React sanitization available on the server path.
  • SSRF prevention. Video proxy validates the upstream URL against a strict allowlist — https://video.twimg.com only — before any fetch is made.
  • Helmet.js + CSP. No inline scripts, no eval, strict connect-src directives.
  • HSTS preload. 31,536,000s max-age with preload flag — browsers won't attempt HTTP at all.
  • 4-tier rate limiting.
    • Tweet reads: 120 req/min per IP
    • Page structure: 200 req/15min per IP
    • Video streaming: 600 req/min per IP+URL
    • Admin auth: 10 req/15min (brute-force hardening)
  • Staging noindex. X-Robots-Tag middleware detects the staging subdomain and sets noindex globally — prevents accidental indexing of the staging environment.

Lessons learned

01

Nginx bind-mount inodes

Editing nginx/default.conf with a file editor that replaces the inode (vs. in-place write) breaks the running container's mount silently. Fix: docker compose up -d --force-recreate nginx after every config edit. A sharp edge worth documenting for any team.

02

SSR route ordering is load-bearing

In Express, SEO routes must be registered before express.static(). Getting this wrong silently serves the SPA shell to crawlers — no error, just invisible to Google. There is no warning.

03

LLM JSON is not JSON

Model outputs contain markdown fences, trailing commas, smart quotes, and unterminated strings at non-trivial rates. A 7-step recovery parser is not over-engineering — it's table stakes for production AI pipelines.

04

Noindex is a feature, not a fallback

Enforcing noindex programmatically on thin pages (few stories, unpublished content, staging) is cleaner and more reliable than robots.txt patterns. It's also closer to how Google actually interprets the directives.

05

Rate limiting needs to be semantic

A single global rate limit on all endpoints creates false positives for legitimate video-scrubbing behaviour. Separate limits per endpoint type, keyed correctly (IP for auth, IP+URL for video), solve this without blocking real users.

Next steps

  • Full-text search across the story archive (Atlas Search or Elasticsearch).
  • Web Push API for breaking news editions.
  • Automated updatedAt propagation to sitemap <lastmod> — currently manual on editorial patches.
  • Edge caching with Cloudflare Workers for the highest-traffic story pages — moving hot content to the edge entirely.
  • Structured author attribution in JSON-LD author field for E-E-A-T signals — schema is ready, surface is not.
Get in touch → Other projects