Recommendations for MongoDB Driver and Schema solutions?

Attempting to fan a personal :fire:of inspiration today →

what would it take to use MongoDB with the Redwood Tutorial (as a replacement for Prisma + SQLite/Postgres)?

Looking for tooling recommendations

  • data modeling & schema validation
  • CRUD
  • Hosting: I’m already sold on MongoDB Atlas (probably just connect locally and skip local setup for now)

Option 1: Do it all with MongooseJS

It’s been years, but I have prior experience with Mongoose ODM. So that could work.

Option 2: MongoDB NodeJS driver +

The native driver looks like it’s come a loooonggg way. So maybe a setup like:

  1. Native Driver CRUD
  2. Data Modeling / Schema Validation
    • Native Schema Validation: could this be done in my project code and “run” similar to how Prisma Migrations are done? I really don’t want to have to use a dashboard to manage the schema
    • Joi looks super interesting for Schema/modeling validation
  3. Indexing: What’s the current state on having to manually manage Indexes… do I need to worry about handling that, too?

Option 3: X

Anything else that’s obvious and I should know about?

1 Like

When I looked into this myself I thought the MongoDB Realm GraphQL API could be a possible solution. You’d likely have the limitations that come along with GraphQL but on the flip side it probably would be fairly simple to spin up compared to a driver. It looks like it’s well integrated with Atlas as well.

1 Like

Yeah, that used to be called “Stitch” and it’s cool product for sure. If I were building Gatsby or Next, I’d just use Stich effectively as my serverless API.

But, if I actually attempt to do this, I want to connect directly and use the Redwood API. Make sense?

Hmm kind of, I’m a little confused by what you mean when you say “use the Redwood API directly.” Do you mean that Realm would basically be the Redwood client making requests and bypassing the Redwood API side?

Ah, sorry about the confusion. I don’t want to have the Redwood API connect to the Realm API that’s connected to MongoDB. I just want to have Redwood API <–> MongoDB.

The best use case for Realm (as I understand it) is as it’s own serverless API for your Client to connect to. So you could effectively just have Redwood Web connect to Realm and drop the need for the Redwood API. This is why it seems like a good API/backend for Gatsby and Next apps.

Cool thanks that definitely makes sense!

1 Like

I"m looking to use it with MongoDB too. Just started today. Did you make any progress on this? I’d love to know how you configured it to work directly with MongoDB.

I will say that one of the issues I’d imagine that’s a huge impediment to what you are trying to do is that the API is setup to use GraphQL as it’s interface to the DB I think and that’s not necessary with MongoDB. I think you’d have to throw out the whole pre-configured API portion of the project, install mongoosejs as you suggested in Option 1 and then connect to MongoDB from there.

Not sure if this would create a bigger mess than either of us would want to take on, but it’s been something I thought was odd from the beginning that they don’t just use the CLI to ask which DB you want. Maybe it’s too much to ask at this stage, but regardless it’s what I’d hoped for.

Hi @misterhtmlcss welcome to the Redwood community! I’m on the Redwood Core Team — happy to help you find your way around and get settled in beyond any DB questions as well. Reach out anytime.

the API is setup to use GraphQL as it’s interface to the DB

^^ this is (thankfully) not correct. For reference, here’s a diagram of Redwoods Architecture. Even though Redwood’s API is a GraphQL endpoint, it’s connecting directly to the DB (by default using Prisma). Swap out prisma with mongoosejs in the diagram and you’re off to the races.

Given that there’s a lot of additional goodness that comes with Redwood’s API structure, I’m just not sure Mongoose is even needed — it’s probably overkill. Thus my curiosity about using the MongoDB NodeJS driver.

Make sense?

1 Like

Having fiddled around myself with connecting Redwood to a database that veers off the happy path I think until you try to build a simple app with both approaches you’ll have a hard time predicting ahead of time what the pros and cons are of different approaches.

I don’t have much experience with Mongoose but the question I would ask is how many things in the core tutorial would you be able to do with just the Node driver versus bringing in something like Mongoose? If the Node driver can get through the same flow without too many extra steps than Mongoose would probably be overkill.

2 Likes

Thank you for the warm welcome. I’ll definitely reach out.

Also I’ll definitely try it. I’m going to spin up a small little todo app using RedwoodJS and Mongoose/mongoDB.

@ajcwebdev you make a great point. I think you always need to consider which tool is best. For me and my typical stack I find Mongoose is usually the best choice so I usually practice everything with it, just so I’m always becoming better with the tool. And yes a todo app doesn’t need Mongoose, but the next app might and I’d like to get that experience under my belt first. Wouldn’t you agree?

Thank you!

1 Like

I have Mongoose experience as well and I’m happy to help contribute to this. Any good starting points? Or just look at how postgres is working and reverse engineer? :joy:

@hyliancoder welcome to Redwood! Admittedly, my enthusiasm for this has not correlated with time and effort. I think a POC is valuable and would get a lot of community interest/feedback/attention. So if you’re interested, I’d be very happy to help as I can!

The main reason for attempting a specific integration is that Redwood uses Prisma for the DB connection/query/migration layer. It’s not required, but it sure is handy. Prisma2 was released earlier this year and so far only supports SQL-based DBs. MongoDB is coming. However, my guess is that we won’t see it until mid-2021.

If you haven’t yet, do take a look at the Redwood Tutorial — especially the “Getting Dynamic” section.

Next up would be looking at some examples of other Redwood DB integrations that don’t use Prisma:

  • This is a (very early) example with Neo4j
  • More recent example with FuanaDB

Sketching out an implementation:

  • Redwood currently instantiates the Prisma Client in api/src/lib/db.js It makes sense to use this for MongoDB client setup
  • In the near-term, we’ll be renaming api/prisma as api/db. Makes sense do use that directory structure for this as well and have schema related files in that directory
  • I’m partial to attempting the native Mongo driver. But if you’re familiar with Mongoose, that makes sense to try first
  • Ideally, there would be a nice “local dev setup” and seamless deploy. But, again, for a POC you could just connect to something like Atlas and not worry about the local setup.
  • Lastly, Redwood :heart:making use of the CLI for workflow, e.g. rw db save... up... seed. I don’t think a POC needs to include going this far, but it might be interesting to consider what the workflow might/could be and, if applicable, add some package.json scripts.
    • note: if you do want to try things like the Mongo native schema solution, this type of CLI/Script workflow could come in handy

Thoughts/questions/interested??

1 Like

That helps! Thanks for taking the time to write that up. I think that’s a good starting place, I’ll let you know if I have questions along the way (I’m sure I’ll have plenty)

1 Like

forgot to say as well, as far as “I’m partial to attempting the native Mongo driver. But if you’re familiar with Mongoose, that makes sense to try first” is concerned, I get you now, you don’t want a api > mongoose > mongo setup if you can just have a api > mongo flow.

I thought Mongoose was the preferred one since it was Option 1 on first reading your comment haha.

I’m completely willing to try native MongoDB first and see if it has any pitfalls. As far as Indexing, you cant type it via the field like you can in Mongoose via:

const User = new Schema({
    name: { type: String, index: true }
})

and instead has to be done on the collection level:

const createAscendingIndex = function(db, callback) {
  // Get the users collection
  const collection = db.collection('users');
  // Create the index
  collection.createIndex(
    { name : 1 }, function(err, result) {
    console.log(result);
    callback(result);
  });
};

so it would need to be something that ran after the model/collection that looks for the indexing happening in the api, and if it does, then stick that function at the end of the model
or
make a big function of all indexes needed schema-wide added at the end.

but due to separation of concerns, im more partial to adding it per model/collection, but obviously just my opinion on that part :joy:

^^ yes, exactly! And sorry about the confusion from the number list – prioritization was not (intentionally) implied :man_facepalming:

Oh my, that’s some ugly code for creating the index. Agreed about adding per collection. If this ends up having some legs, maybe we could think about adding a helper/utility exported from db.js

All that said, go for it! One idea for a POC is to see how/if Mongo could be used with the Redwood Tutorial specifically for blog post CRUD. But maybe that’s too ambitious for a first step.

Lastly, I’ll be offline this Friday until Nov 12th. Will catch up when I’m back. No urgency or pressure on anything. Just didn’t want you to be waiting on me and wonder what happened.

1 Like

All that said, go for it! One idea for a POC is to see how/if Mongo could be used with the Redwood Tutorial specifically for blog post CRUD. But maybe that’s too ambitious for a first step.

If someone wanted to scale that back I’d recommend just setting it up so there’s some really simple seed data in the database and Redwood is just rendering that out so you’re only worrying about reads. That’s how I started out with FaunaDB and now we’re expanding out and figuring out how to do the rest of the CRUD actions.

2 Likes

I appreciate all the answers and the heads up!

1 Like

@hyliancoder hi there! I’m back in the saddle and checking on things. Available next week if there’s anything you want to kick around. No pressure either way.

I think using GraphQL Compose with the Mongoose plugin could be a great way to support MongoDB. Being able to lean on conventions to generate the schema and resolvers from Mongoose models could be very helpful in reducing boilerplate, while being flexible enough to allow the types and resolvers to be tweaked as needed. Gatsby is also using GraphQL Compose to generate their types.

1 Like

Hi @corydeppen Welcome to Redwood! And thanks for giving this thread a much-needed nudge.

Those options look interesting for sure. Just took a quick look and curious about a few things:

  • is this in addition to creating a data model with mongoose or could it possibly replace it? I saw that it utilizes the model if you already have but wondering if you could just use this as well for model + SDL + Types
  • do you know how this handles creating/composing the SDL?
  • Redwood handles resolvers for you when creating it’s GraphQL API function. Wondering out loud how/if this is compatible. Do you have any initial guesses?