Introducing RedwoodRecord

Several weeks ago I had a vision. A vision of easier access to data. Simpler syntax to access my database. Today, that vision becomes reality.

With the release of 0.39/1.0.0-rc.1 we’re including an experimental package called RedwoodRecord. It’s modeled off of ActiveRecord from Ruby on Rails. It’s an ORM that wraps Prisma and provides class-based access to your data. We’re calling it “experimental” because we don’t really know how the community is going to react to it, and the API isn’t fully baked. We’re trying some things out and need feedback to see where we should go with this library.

You can check out the official docs here, but here’s a quick guide to getting started.

First, add to your package.json:

yarn add -W api @redwoodjs/record

Next, generate a “model” class which will represent a single table in your database:

yarn rw g model User

(Take a look at api/src/models/index.js to see what was added.)

You now have a model at api/src/models/User.js. Before we can start using it, we need to run one more command which will turn your schema.prisma file into a cached JSON file, and create the index.js file in the models directory, adding a bit of config for the models:

yarn rw record init

For now you’ll need to run this command any time you change your schema, or after creating new models.

Now, you can update your users service to use your model instead of raw Prisma:

// api/src/services/users/users.js

import { User } from 'src/models'

export const users = () => {
  return User.all()
}

export const user = ({ id }) => {
  return User.find(id)
}

export const createUser = async ({ input }) => {
  return User.create(input, { throw: true })
}

export const updateUser = async ({ id, input }) => {
  const user = await User.find(id)
  return user.update(input, { throw: true })
}

export const deleteUser = async ({ id }) => {
  const user = await User.find(id)
  return user.destroy({ throw: true })
}

Since models are just classes, you can add your own functions/properties to the class and they’ll be available anywhere a user is. Let’s say you store firstName and lastName separately in your database, but you also want to display a fullName. Rather than write a function somewhere else and import it when needed, just add it to User:

// api/src/models/User.js

import { RedwoodRecord } from '@redwoodjs/record'

export default class User extends RedwoodRecord {
  get fullName() {
    return `${this.firstName} ${this.lastName}`
  }
}

And now whenever you want someone’s full name:

const user = User.find(123)
user.fullName 

You can also replace the object returned in getCurrentUser to be an instance of your model (dbAuth is used in this example, so session is just an object containing the id of the logged in user):

// api/src/lib/auth.js

export const getCurrentUser = async (session) => {
  return await User.find(session.id, { select: { id: true, email: true } })
}

Now, currentUser on the api side will be an instance of RedwoodRecord, so you can get to related models directly through your user. Here, we’ve also created a Post model and now we can get only those posts that the current user owns:

// api/src/services/posts/posts.js

export const posts = async () => {
  return context.currentUser.posts.all()
}

export const post = ({ id }) => {
  return context.currentUser.posts.find(id)
}

export const createPost = ({ input }) => {
  return context.currentUser.posts.create(input, { throw: true })
}

export const updatePost = async ({ id, input }) => {
  const post = await context.currentUser.posts.find(id)
  return post.update(input, { throw: true })
}

export const deletePost = async ({ id }) => {
  const post = await context.currentUser.posts.find(id)
  return post.destroy({ throw: true })
}

Start playing with it and let us know what you think! And check out the full docs for more syntax: RedwoodRecord | RedwoodJS Docs

Caveats

RedwoodRecord is not currently typed. The attributes on a model are created dynamically based on the object returned by Prisma, so I’m not even sure if it can be typed in its current iteration. If you have any insight into helping add Typescript to this library, get in touch!

10 Likes

First the framework, then dbAuth, now this :exploding_head: Thank you Rob, looking forward to getting to try this out!

(I should have bit the bullet and learned Rails, I had no idea it had so many neat things up its sleeves)

5 Likes

Looks neat, like prisma with training wheels. Is the goal to improve developer experience, by getting them working with data sooner in their journey?

  1. How quickly will developers out-grow RR, or is it intended to be a full-featured API?
  2. Will it feel like helpful “training wheels”, or a distraction from just learning prisma.
  3. Every time we diverge from a single path, like we did with the database, it creates a lot of confusion for newcomers. Is it possible to avoid multiple paths, by making RR the official Redwood Way ™?
1 Like

Yeah assuming this doesn’t turn out to be a dead end we’d probably make it the default interface to accessing the database, and then you can drop down to Prisma if there’s something you can’t accomplish through this interface. If it became as full-featured as ActiveRecord is in Rails then you could do 99% of anything you need to do through here. Time will tell.

It’s “experimental” so we’re not promoting it anywhere to avoid confusing newcomers. The tutorial and all other docs (besides the RR doc itself) only discuss Prisma.

How/why could this become a deadend? SQL itself is based on relational algebra, and I know absolutely nothing about the theoretical mathematics behind it. For example, turning these equations into Javscript:

:man_shrugging:

So I worry that I can only get so far in this implementation before not having a Ph.D in computer science will become a problem.

Not many people even dip down to this level, but the gem behind the database queries in ActiveRecord is arel, which is a layer between the familiar AR interface and whatever connector you’re using to actually talk to the database: File: README — Documentation for arel (9.0.0) THAT is where the relational algebra stuff is expressed into chainable functions and it twists my mind into a pretzel.

3 Likes

:eyes: .

3 Likes

I worked with Rails for a few years and I love Active Record. Should I incorporate RedwoodRecord into the app I’m working on now? Or, is this going to fade away due to lack of interest. I think you might see that Rails developers would find this as a very nice and familiar feature. Frankly, I’m still trying to see the benefit of graphql. I’m relatively new to React and Web3.

2 Likes

We don’t have any plans to remove it! It’s pretty self-contained in the codebase, so even if no one other than me used it, we would keep it around. :wink: I’m just not sure how much active development it’s going to get if most people are happy wrangling the database through Prisma directly. Seems a little too low-level for productive app work if you ask me, but ignorance is bliss: these people don’t know what they’re missing!

We haven’t really made a concerted effort to promote it outside of the forums here. I’d like to do an introductory video or webcast to really show folks what it can do. But I don’t know that it’ll become as full-featured as Rails ActiveRecord unless someone that really knows the ins and outs of the theories behind relational algebra takes a crack at it.

Ok. I’ll give it a try and integrate it into my app. I’ll make 2 or 3 models and see how it works. I’ll let you know what I think.
J

2 Likes

This feature sounds neat. I tried using it because it liked the way it handles the validations insead of doing them directly on the service mainly because it groups the validation errors per fieldName so when I get those errors in the web I can easily show them on the corresponding field.

To contrast this, the current validate method used in the services validates only one field at a time and returns a promise that is rejected if the validation throws errors, this means that in order to return the validations for all fields the same way the RedwoodRecord does you need to wrap each validate in it’s own try catch and merge the errors at the end to throw a single error object.

Most errors should have been caught by the client side validations for a better user experience and de backend validators should be more of a safe-guard so maybe returning the errors for the first field that returns errors only might be fine but still the RedwoodRecord gives us a better dev experience.

The issue I am having right now is that the recods lib is in JS and I am using typescript for my project so even though my record extends RedwoodRecord, it says that for example build is not a function of my record because RedwoodRecord itself is not typed.

Would be really awesome if it was typed. Completely understand that it should probably not be a priority right now since v1.0 is getting ready to be released.

I think going forward this feature is sth that is going to be useful so :+1:t2: from me to continue the development of it. I might be initerested in lending a hand with the TS conversion if that is a thing we want to move forward with.

3 Likes

Thanks, and welcome! Sorry about the lack of TS…I hate it with the fiery passion of a thousand suns, but most people seem to like it! Going in I had no idea if they even could be typed (properties are created on the fly using defineProperty based on what comes back from the database). But @orta theorized that it could be typed doing something like this: typescript.org (I have no idea what any of that code means)

If you wanted to try implementing that we’d be forever grateful! We do have a persistent running type generation service now, so maybe something could be hooked into that which would generate types on the fly if/when Prisma models change? That thing is here: https://github.com/redwoodjs/redwood/blob/main/packages/internal/src/generate/watch.ts

My only requirement is that if I generate a JS project, I don’t want to see any remnants of changes that had to be made to satisfy some arbitrary TS requirement. I like the interface and generated code just the way it is, thank you very much! :slight_smile:

2 Likes

Ok. I set it up in a new test project. And. It works! I like it and will use it in my next project. Thank you very much!
John

Hey @rob would love to talk about where RR is at now and what would help flesh it out.

Personally I’d love to get scopes, used those heavily in Rails, and permissions (maybe a pundit clone) builtin at the RR level would be interesting.

Also RR native transactions!

Also I can probably help with the arel problem. That kinda stuff is usually easy for me once I get my head wrapped around it.

1 Like

Hello! You’re more than welcome to take a look at expanding it. I haven’t really touched it since the initial experimental launch. You were having trouble with v6? Nothing about RR itself has changed, but it’s possible that the hook the triggered the schema re-scan got removed somehow? It’s been so long since I worked in that code…

I’m not sure why it’s not picking up the function. It’s the only one that seems to have an issue. It simply doesn’t recognize the task function as a function.

My guess is whatever code binds the task functions to the CLI got changed, but I haven’t dug into how any of that works.

1 Like

I looked through the history of that script and I don’t see any change that would have caused what you’re seeing…

Here’s the file: https://github.com/redwoodjs/redwood/blob/main/packages/cli/src/commands/generate/model/model.js

And here’s the call to parseDatamodel(): https://github.com/redwoodjs/redwood/blob/main/packages/cli/src/commands/generate/model/model.js#L70

As long as that package is installed I don’t see why it wouldn’t work? Maybe you can insert some debug into the source in node_modules and see if something is erroring out between doing the async import and calling it?

Here’s the actual function: https://github.com/redwoodjs/redwood/blob/main/packages/record/src/tasks/parse.js#L30-L91

And it’s exported from the package’s index.js here: https://github.com/redwoodjs/redwood/blob/main/packages/record/src/index.js#L8

I did a quick grep for parseDatamodel in the node_modules and it looks like there’s a difference in how the build output is calling that function between record/init.js and generate/mode/model.js .

record/init.js

var _record =  = require("@redwoodjs/record");
await (0, _record.parseDatamodel)();

and

generate/model/model.js

const {
  parseDatamodel
} = await import('@redwoodjs/record');
await parseDatamodel();

And it’s the latter call that fails.

The difference in the source code is that record/init has the import at the top as normal, where generate/model has it inside the subtask structure and that seems to yield different results after being built.

Dumping the parseDatamodel function to the console returns undefined

And if I convert the built version of generate/model to match record/init it works. So it seems like the way it’s being imported is not working, probably something in the build toolchain changed/updated and broke the transpiling?

I’ll throw an issue up on GitHub with my findings and shoot over a pull request shortly. I’m not sure how to build/test any of this yet, but at least I can get the ball rolling.

1 Like

Huh…when going through the history, I originally wrote a standard import at the top of file. A few months later someone else on the team changed it to an async one. I assumed that was so if you weren’t actually using @redwood/record it wouldn’t include it in the bundle. People have been using RR since then, and it’s been fine. However, in v6 we switched to making vite the default bundler (instead of Webpack).

Maybe those async imports are handled differently now? I’ll ask the team and see if anyone has some insight…

Ah ok, that makes sense with the vite change. I was on web pack when I first got setup and did the 6.0 upgrade.

I have an issue and PR up on hub. The PR should ‘fix’ the immediate issue of it not working.

For the async import part, that would need to go in record/init to match as well? Otherwise that would pull in @redwoodjs/record right?

1 Like

I think we found the issue, I replied to the PR! Let’s move discussion over there…