Memory use in Redwood v6

[eraoul]

I recently upgraded my project from the v6 RC to the official 6.0.6 version. I’m hosting my project on Render.com and I’m using the free tier while I prototype, so I have a tiny CPU/RAM etc. But in any case, I’m seeing intermittent out-of-memory errors, and I don’t think anything changed on the frontend aside from the upgrade. Error message:

Server unhealthy

Ran out of memory (used over 512MB) while running your code.

512MB is tiny, to be sure, but I’m curious if there is something larger than expected in 6.0.6 vs the v6 RC.

[thedavid]

Hmm, the only time I’ve seen this memory limit error is during deployment. For hobby/dev projects, based on my experience, agreed you should be ok. But

Do try the latest patched version for v6, v6.0.7 and let me know if there’s any improvements. The performance fixes are mostly for AWS Lambda infra, but maybe they were also affecting Render if just subtly.

[eraoul]

That’s true, I’ve only seen it in deployment so far as well.

I tried v6.0.7 tonight and still ran into the same issue though (on the Render.com free tier, again) – but on retry the deployment was successful.

[eraoul]

I’ve upgraded to 6.2.0, but I’m still having memory errors during deployment. I think it’s due to DB migrations though. The log has errors like this, where I think the exit code 137 is the out of memory error.

I haven’t been making any DB migrations recently; I think these are all old from initial schema development. Any ideas if this memory error is expected, when I’m running with only 512 MB RAM?

I could upgrade to 2GB on render.com but I wasn’t sure if this was the problem or if I’m doing something wrong. Also the prices for a mere 2GB of ram seem steeper than I expected, so maybe I should look around for other deployment options, but that’s a different story!

Sep 10 11:09:14 PM 2 migrations found in prisma/migrations
Sep 10 11:09:14 PM
Sep 10 11:09:14 PM
Sep 10 11:09:14 PM No pending migrations to apply.
Sep 10 11:09:23 PM Installing plugin “@redwoodjs/cli-data-migrate”…
Sep 10 11:09:39 PM Killed
Sep 10 11:09:39 PM /opt/render/project/src/node_modules/execa/lib/error.js:60
Sep 10 11:09:39 PM error = new Error(message);
Sep 10 11:09:39 PM ^
Sep 10 11:09:39 PM
Sep 10 11:09:39 PM Error: Command failed with exit code 137: yarn rw dataMigrate up

[arimendelow]

Ah yeah I have the same problem with my Build Competition’s project deployment - didn’t want to deal with it so for the month I’m on the 2gb plan, but it is quite expensive for a hobby project.

Redwood shouldn’t be running out of memory with only one user, definitely seems like something is off. I haven’t had this issue with a larger project with more users that’s still on v5 - haven’t wanted to upgrade yet specifically because of this issue.

I haven’t tried this out yet, but this is what the Render team suggested:

To prevent NodeJS’s underlying V8 engine from allocating more heap space memory than what your service allows for, try setting an NODE_OPTIONS environment variable with the value --max-old-space-size=1536 . Change the value to 920 for a 1GB instance and 460 for a 512MB instance. These values are just rough guides, you may be able to increase it or need to lower it depending on how your application uses its memory.

If this doesn’t help, we have a blog article on how you can remoting debug NodeJS applications here: Remote Debugging with SSH and VS Code | Render

[eraoul]

Thanks for the suggestions here. I tried several values of --max-old-space-size but never was able to get this working.

I’m going to try to switch over to Vercel and see if that works any better. My only concern is that its Postgres and Python serverless options are both in “Beta”. I have a separate python backend API so I really need python, but I think serverless might be better anyway if I can get it working.

[razzeee]

I’m only aware of tests eating into memory quiet badly, but that might be connected.

[arimendelow]

Has there been any updates on this? I’m running into the same memory problems in production, even on 6.3.2.

Something is definitely very wrong here - we shouldn’t need so much memory to host a basic RedwoodJS app. The below is with basically no usage:

[Josh-Walker-GM]

Hey @arimendelow, I agree this seems like a lot of memory to run a basic app. Can you give me some more idea of exactly what is running and the complexity of the app. If I can reproduce a comparable app then I can start to narrow down where we’re using lots and try to get it down.

Is this with yarn rw serve? Does this also take into account the yarn rw build step? Should I attempt to reproduce a similar size of graphql models, cell/page count, etc.

[arimendelow]

Hey Josh! Thanks for helping out on this

You can actually repro by just deploying this project yourself - it’s my build competition project.

I recall talking to others at the conference that ran into similar issues when upgrading from v5 to v6 - it seems that there’s some dependency(s) that was upgraded that’s causing this.

You can see the Render config in the repo - https://github.com/shmobs/yurtle/blob/main/render.yaml

Let me know if there’s any other info I can provide! I recognize that this is just one datapoint - I haven’t yet upgraded my larger Redwood project (Spoonjoy) to v6 yet as I’m worried I’ll run into the same issue and haven’t wanted to test that out

[Josh-Walker-GM]

Oh perfect! I’ll check this out and see if I can spot where the large memory usage is coming from and try to compare between v5 and v6.

Your build competition entry was awesome btw!

[arimendelow]

Awesome, thanks so much!!

Awwww thank you!!! And it was lovely meeting you at the conference

[oreid]

Adding to what @arimendelow has said I can report that this occurs for brand new apps which aren’t complex at all. It seems using v6 you aren’t able to run a brand new Redwood App on Render’s free tier. I’ve upgraded to a tier with more RAM and it works fine, but something is definitely using too much memory. If it’s useful I can get some metrics on how much memory was required to build successfully.

[max-scriptt]

I’ll cosign this. Ran into the same issue on my first deploy as well. I ended up upgrading to the $25/month plan to run it. After that, it worked great.

I believe this happened during the migrations, but don’t quote me on that. I have about ten tables that needed to be created.

[cjReimer]

We have two render api instances… The first is a small app that is on a free plan with only a couple of GraphQL endpoints. In order to get that running, we had to set an environment variable
NODE_OPTIONS with a value of --max-old-space-size=460.

As for the other api instance, it’s a larger application that we still have running on one of their obsolete 1MB Starter Plus plans. The issue with memory usage appears to be related to deploy only.

Note that render.com has a new command available in the settings that are not currently used by Redwoods default redwood deploy. We just tried the following changes, and it might be making a slight improvement in the deploy memory usage: Effectively, we are doing the prisma migrate slightly sooner. In looking at redwood’s deploy command, this should be running the exact same sequence, but perhaps there’s some nuance to it.

Pre-Deploy Command: yarn rw prisma migrate deploy
Start Command: yarn rw deploy render api --no-prisma

From what I can notice, the render.com metrics for memory usage don’t always appear to chart the peaks, so we’ll have to watch this over time.

[deep-randhawa]

Hi - I’m running into the same issue, but I’m on v5 still…

This is a Production issue for us; any ideas on how to resolve this (short term and long term would be much appreciated here).

From render logs, seems like Render keeps running out of memory. Our application just has some simple GraphQL endpoints, nothing I would consider “risky” that should be taking up more memory that should just break the entire application (of course, happy to debug this more and be corrected here).

<--- Last few GCs --->
INFO
INFO[75:0x692c5c0]   908092 ms: Mark-sweep 4035.0 (4134.8) -> 4020.6 (4136.6) MB, 1591.9 / 0.0 ms  (average mu = 0.136, current mu = 0.010) allocation failure; scavenge might not succeed
INFO[75:0x692c5c0]   909682 ms: Mark-sweep 4036.6 (4136.6) -> 4022.4 (4138.6) MB, 1574.6 / 0.0 ms  (average mu = 0.077, current mu = 0.010) allocation failure; scavenge might not succeed
INFO
INFO
INFO<--- JS stacktrace --->
INFO
CRITICALFATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
INFO 1: 0xb87bc0 node::Abort() [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 2: 0xa96834  [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 3: 0xd687f0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 4: 0xd68b97 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 5: 0xf462a5  [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 6: 0xf471a8 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 7: 0xf576b3  [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 8: 0xf58528 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO 9: 0xf32e8e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO10: 0xf34257 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO11: 0xf1542a v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO12: 0x12da78f v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/opt/render/project/nodes/node-18.18.0/bin/node]
INFO13: 0x170a079  [/opt/render/project/nodes/node-18.18.0/bin/node]

[n1tsua]

Hi there! I just want to confirm that I am also running into this issue and was wondering if this is currently being prioritized. I just started a new project and decided to use Redwood (I think it’s awesome!), but am using Render for deployment and am consistently getting Out of Memory failures when trying anything other than a 2GB mem machine.

I have tried some of the options listed on here, but was never able to get everything to run on the 512MB machines. Here’s a look at the memory consumption. The peaks are during deploys.

Below are the commands for build, startup, and start:

  buildCommand: yarn && yarn rw build api
  preDeployCommand: yarn rw prisma migrate deploy
  startCommand: yarn rw deploy render api --no-prisma

[danny]

Hi @n1tsua, thanks for reporting this.

Can you confirm if the OOM issues are happening during deploy? i.e. is the build machine running out of memory or is it that the machine that’s running your application that’s running out.

@Josh-Walker-GM is going to look at this today, and is being prioritised.

Thanks,
Danny

[n1tsua]

Yes, can confirm this is during deploy, the machine halts after running the deploy command.

[arimendelow]

in my situation, it runs out of memory running the application as well