Should a storage abstraction be a part of Redwood?

The lack of a persistent filesystem is brought up as a drawback of the Jamstack
and it is a clear hurdle for people to conceptually overcome.

I think Redwood would benefit from helping people overcome this issue by suggesting or integrating a solution. While people can write coupled code specific to one of the storage platforms (e.g. Filestack, AWS S3) this requires quite a bit of boilerplate, requires people to think of this solution to non-persistence themselves and lacks the standardised and cleaner code of an integrated solution. Further, it makes them hard to swap and use different providers - e.g. if I want to use S3 in production but a local folder in development.

Rails offer Activestorage and there could be potential for a similar offering in Redwood.

A nodejs package that could potentially be used is flydrive:

While it doesn’t offer the image specific APIs Cloudinary, Filestack - the code for these services tends to be less generic and so potentially would be less of a candidate for a more abstracted solution. The use case I am thinking more of is storing custom log files, non-media file uploads, etc.

Any ideas on whether this is a good idea or not / whether it fits with Redwood’s value proposition?

1 Like

@georgewritescode Storage is defined a piece of the complete stack that — as you note — is terribly useful but also takes a bunch of boilerplate to get up and going.

At the moment, I don’t think RedwoodJs has a strong opinionated suggestion yet but one leading option right now is Supabase storage Storage | Supabase

You can use this both on the web side via their sdk client and api side.

It wraps and abstracts out the storage provider and keeps all the info for that file in Postgres.

If you need a file system to stores logs or other non ephemeral things, then that YouTube a different solution.

Storage is needed

2 Likes

I think if we were to start from first principles, then S3 is a great standard. The API is copied and compatible with many providers: Backblaze, Azure, DigitalOcean, IBM Cloud Object storage, etc. So you won’t be tied into a single provider, and it offers you a great amount of flexibility.

I think to start off with RedwoodJS could create a example site to show how it’s done, like Next has done over here: GitHub - leerob/nextjs-aws-s3: Example Next.js app to upload photos to an S3 bucket.

This example above doesn’t feel very “boilerplatey,” the ability to upload things is ~15 lines, and a few lines of configuration for the service.

Maybe we can build up the example for a few more things that people want to do with files and use that as a jumping off point for thinking about a Redwood like API.

1 Like

Thanks @dthyresson and @peterp for your responses.

I think this makes sense as an approach.
I agree an example as a starting point is a good idea to start with.
Over the medium-term, I think it would be good to have a Redwood API to improve the developer experience for those who are new to the Jamstack and this lack-of-persistent filesystem challenge (even for temporary files).

Similar to Auth, perhaps we could have a Redwood API/Interface, provide the solution for S3 compatible filesystems, but also allow people to create 3rd-party providers for other filesystems.

2 Likes