I’m brand-new to JavaScript and I’ve been playing around with my first serious web project, using Redwood (I’m more of a python/C++ machine learning engineer, plenty of computing experience but I’ve avoided any serious web development so far).
I’ve defined a couple models in api/db/schema.prisma and I’m using them successfully in a couple test pages of my site. Now, I’m building a separate backend service in python for some ML code. Two questions:
How can I access data from the GraphQL DB created by Prisma directly from the python service? Is prisma-client-py the way to go? If so, any tips on how I wire up my python prisma client to my Redwood/prisma-generated DB? There’s magic going on behind the scenes that I don’t understand well that creates the DB, so I don’t know what I need to do to keep the python API in sync with the Redwood Prisma DB. Hope that question makes sense.
I want to do a long-running computation on the backend, and send results periodically to update the view in a React component. I’m thinking of doing this via websockets. I’ve also heard of a GraphQL websockets interface. Does that make the most sense to use here since since we’re also using GraphQL on the client? Is there any redwood-specific integration I need to know about that would help support long-running computation with periodic async updates to the frontend?
Thanks for any tips! I’d like to use this in a serious public-facing site so I’m hoping to get the design right for this prototype.
First off, there’s no “one way” to do this. I do think you’re confusing some of the app boundaries and how DB <> Prisma Client (queries) <> GraphQL API work together. That said, some thoughts:
Via the Python app (service), you can either a) connect directly to the DB or b) use the Redwood GraphQL API or c) create a specific, new API endpoint via a custom Redwood Function.
“Polling” is much easier and doesn’t require infrastructure. Unless your app requires true realtime (i.e. chat), polling works 90% of the time with much less overhead.
Lastly, I’m reading a bit between the lines but I think you should check out Inngest for a Cron/Jobs/Events service that integrates very nicely with Redwood. You could likely use Inngest as the bridge between your Redwood and Python services. Note: the integration package (below) is still experimental, but the Inngest service is it’s own product and can be managed manually if needed:
Hi @thedavid - thanks so much for the quick reply and the ideas. I’ll look into the things you suggested. Yes, I think I 'm still confused about the app boundaries and trying to sort that out in my head. Two followups in case you have another moment to reply:
For # 1 I liked idea b, using the Redwood GraphQL API, since I’ll be wanting to grab data in the same way that I will on the frontend; I’ll just be using it to run some computations instead of rendering UI. That sounds simplest. I don’t think I know the best way to discover/access this API from python, however. Could I simply use any old python GraphQL library for this?
Polling: maybe that’s the way to go. Let me give more context: when I said “long-running” I actually meant on the order of 5-10 seconds (just long-relative to ms-scale UI refreshes). I’d like to update the UI frequently as this runs both to make the app look responsive and to show a cool animation each time the data updates (using d3). Do you think polling still makes sense in this context? I’m thinking I’ll just grab an update every 0.5 seconds or so while it runs. I assume Inngest is what I’d want for actually-long-running on the order of minutes/hours/days, but not this shorter-scale thing. (I’ll probably have use for Inngest elsewhere in the project though, so this sounds super useful, thanks!)
GraphQL API + Client
We use Apollo GraphQL Client for Redwood. Highly recommend checking out the Python version:
(^^ older article but seems like a good overview)
Long Running Process + UI Updates
Ah, that makes sense now. I focused on the “long running process” part (for which I’d still suggest Inngest to manage things if applicable).
Realtime has a lot of overhead. So you might want to look into services like Pusher.
From the Redwood Web component, you can use the built-in Apollo Client to poll. See this doc on pollInterval. Conceivably, you could poll the Redwood GraphQL API (assuming the background process is updating the DB) OR you could poll a GraphQL API from you Python service.
Additionally, you could set up a realtime stream between your Python service and Redwood Web component(s). Here’s an example experiment for that case: WebSockets with RedwoodJS [experiment]