Hi @atyler some years ago (maybe 8 or so … wow … early Netlify days) I built an app using Contentful and Middleman as a static app deployed to Netlify. It grew to render tens of thousands of pages on each build. It was taking 6+ hrs and maxed out Netlify’s build memory – even on an enterprise plan.
I did a number of optimizations because as you said, not all the content changed.
In fact you can read about if here as one of my early interactions with Redwood from 2020:
I got builds down from 6 hrs to 90 mins to 15 mins to 2 mins. But it was quite a bit of work and acrobatics.
I was caching queried data, caching generated pages, and only re-generating new or recently updated content with multiple steps. It was involved, but helped.
However – the real solution was the 2 min version; make it a React app with an api and a Netlify function to be the api endpoint – a crude Redwood :).
For me it was better (because I knew form data analytics that most traffic was for recent content) to have:
- dynamic data fetch
- generate and publish full sitemaps and such generated for SEO (bulk of build time, besides code)
- use Netlify prerender for SEO unfurling
The upcoming Redwood SSR will make this much easier since don’t need to do the Netlfy pre-render service.
Thought I’d share as someone who did render 30k+ pages.
Do let us know how you get on.