Intermitant 502 Runtime.ExitError

Get Help and Help Others
1

Hey, I figured now would be a good time to ask for help, While I have a few ideas of things to explore, they feel like grasping at straws at this point.

Basically I’m getting 502s from graphql queries. It did happen after a deploy, but there were no changes to the API so I’m not sure how it broke.

The return errors as well a what’s logged in the neflify functions log is identical, and pretty cryptic:

5:20:21 AM: /var/lang/bin/node[8]: ../src/node_http_parser_impl.h:529:static void node::{anony    mous}::Parser::Initialize(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[3]->IsInt32()' failed.
5:20:21 AM: 1: 0x5640b570b1a3 node::Abort() [/var/lang/bin/node]
5:20:21 AM: 2: 0x5640b570b232  [/var/lang/bin/node]
5:20:21 AM: 3: 0x5640b5729564  [/var/lang/bin/node]
5:20:21 AM: 4: 0x5640b592e0a6  [/var/lang/bin/node]
5:20:21 AM: 5: 0x5640b592fde1 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/var/lang/bin/node]
5:20:21 AM: 6: 0x5640b61ce519  [/var/lang/bin/node]
5:20:21 AM: Duration: 378.32 ms	Memory Usage: 143 MB	
5:20:21 AM: RequestId: 7f92e612-5fb5-41eb-9b5c-ac2bb78e9851 Error: Runtime exited with error: signal: aborted (core dumped)
Runtime.ExitError

Easy way to replicate it is by with the following query at https://cadhub.xyz/.netlify/functions/graphql

query {
  parts {
    title
  }
}

root resolver for that is (https://github.com/Irev-Dev/cadhub/blob/main/api/src/services/parts/parts.js#L10)

The weirdest thing I think is that the issues seems to be intermittent, It mostly fails, but every once and a while it works :woman_shrugging:.

Ideas that I’m yet to try are

  • add some logging to my resolvers to get some insights (however I think this might be an integration issue with amazon, is the error coming from their load balancer, in which case this is unlikely to help)
  • I thought about setting up connection pooling for my heroku/postgresql db, but I doubt this will do anything since I wouldn’t have hit the 100 connection limit yet as There are no users, just me so far.

Any information I should add? Tips or trouble shooting ideas much appreciated.

2

Actually come to think of it. I should try update to V0.20.
I need to do anyway and it might magically fix this issue.

3

I had this issue starting yesterday morning – and you’re the second person today to bring it up. Seems like something may have changed on Netlify’s end to make this more prevalent. It seems that this is a bug from an older version of Prisma, but upgrading Prisma to 2.9.0 (which Redwood 0.20.0 does) fixed the issue for me.

Also, if you run into any build errors when you deploy your upgrade to 0.20, you might want to take a look at this thread in which @thedavid was nice enough to help me out.

4

Awesome, Thanks @tctrautman,
I’ll upgrade and see how I go :crossed_fingers:

1 Like
5

I started getting this same thing. I haven’t updated a function is 2 months now but started seeing this today:

Runtime.ExitError
12:38:06 PM: /var/lang/bin/node[8]: ../src/node_http_parser_impl.h:529:static void node::{anonymous}::Parser::Initialize(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[3]->IsInt32()' failed.
12:38:06 PM: 1: 0x55dcd55741a3 node::Abort() [/var/lang/bin/node]
12:38:06 PM: 2: 0x55dcd5574232  [/var/lang/bin/node]
12:38:06 PM: 3: 0x55dcd5592564  [/var/lang/bin/node]
12:38:06 PM: 4: 0x55dcd57970a6  [/var/lang/bin/node]
12:38:06 PM: 5: 0x55dcd5798de1 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/var/lang/bin/node]
12:38:06 PM: 6: 0x55dcd6037519  [/var/lang/bin/node]
12:38:06 PM: Duration: 309.80 ms	Memory Usage: 121 MB	
12:38:06 PM: RequestId: 0264b9c1-1160-433e-a01b-bb1880ad7b51 Error: Runtime exited with error: signal: aborted (core dumped)
Runtime.ExitError
6

@rob Is this on a v0.20 project?

7

v0.20 worked for me too :ok_hand:

8

Nope, 0.18.0. But I haven’t touched that function in two months, no idea why it would suddenly start failing.

I wonder if 0.20.0 is really the fix, or just the fact that it’s getting redeployed kicks something into gear over at Netlify’s side. Kinda like having to restart after a spinning beachball of death. :slight_smile:

9

I wonder if 0.20.0 is really the fix, or just the fact that it’s getting redeployed kicks something into gear over at Netlify’s side

My choice of words above was a little misleading – if I’m reading this Github correctly it sounds like the cause is an incompatibility between older versions of Prisma and node 12.19.0 (not a “prisma bug” as I said earlier.)

Maybe something happened a few days ago to cause a subset of Netlify function servers to run 12.19.0? That’s my best guess :confused:

10

Ahhhh interesting. I’ve got a support ticket open with them, I’ll share that GH issue.

In the meantime, I upgraded to 0.20.0 and it worked for me, too!

1 Like
11

Hello, just wanted to jump in here to mention that I saw this happening on my prod environment, where no changes were made since September. No code, or environment var changes, or even a deployment.

Clearly Netlify changed something in their setup in the last week or so, and I suspect its related to the custom headers used in Auth. Raised an issue here: https://community.netlify.com/t/502s-on-functions/26997

Good news is the latest versions are working fine!

1 Like