Thoughts on background jobs

I’m really liking Redwood so far and it makes me wonder about what it’s going to look like in the future.

One of the most important parts of the Rails app at work is Sidekiq. I’m curious as to what y’all are thinking about for background processing. Is this something that is on your radar or is it something that you think of as much farther down the line.

I have used SQS with the serverless framework, but it was definitely not as nice as Sidekiq. As far as a I can tell Netlify Functions are really HTTP focused. It doesn’t look like they really support being called from things like SQS, so that also presents a challenge.

I like the developer-experience focus of Redwood a lot so far and I’m wondering what your take on background jobs would be.

Thanks!

1 Like

This is something that we’ve spoken about, but not really something that we’ve dug into much.

I really like the idea of using a serverless Function as the task runner because the runner can be bundled with only the things that it requires to run instead of the entire application, which is true or not depending on how you compose your serverless task runner.

I think some of the main obstacles:

  1. having a queue somewhere nearby the runner and nearby the database.
  2. preventing access to a runner (since it’s http)
1 Like

If you need to cobble something together today you can create a custom function that does whatever you need and https://www.easycron.com/ to schedule hitting that function URL on a one-time or recurring basis.

We’re talking to Netlify about adding functionality like this, but no promises in the near term. We’ve also maybe started prototyping this functionality ourselves…no promises, but it’s definitely on our minds!

1 Like

I will definitely check out easycron for some scheduled tasks that I plan on running.

I do think there is still a gap for unscheduled background jobs. I’m excited to see what happens next. I will keep thinking about it too.

2 Likes