Running RedwoodJS on Cloudflare Workers

ajcwebdev · April 7, 2021, 11:25pm

A “Cloudflare Worker” is JavaScript you write that runs on Cloudflare’s edge. A “Cloudflare Service Worker” is specifically a worker which handles HTTP traffic and is written against the Service Worker API.

Cloudflare Workers derive their name from Web Workers, and more specifically Service Workers, the W3C standard API for scripts that run in the background in a web browser and intercept HTTP requests. Cloudflare Workers are written against the same standard API, but run on Cloudflare’s servers, not in a browser.

Prisma is currently out of the question

@thedavid

The potential gotchas that keep me from trying:

Prisma binary hates v8 runtime

Some random dependency in @redwoodjs/api hates v8 runtime

Auth provider packages hate v8 runtime

Misc API package dependencies used by random Redwood Project X hate v8 runtime

In general, I foresee it being frustrating if my local dev environment spins up the API in a Node runtime so it’s not until I get to deployment that things break, which is then difficult to diagnose.

So I’d suggest starting with local dev for experimentation → I have ZERO knowledge here:

Is it possible to spin up a local v8 runtime for dev?

Does babel/esbuild support it?

If we can’t do something like this, then I’m not optimistic about creating a workable, um, workflow [Editor’s note: lol] regardless of being able to spin up a test app on Cloudflare using Workers.

Workers do NOT support Node.js.

I believe we can transpile code to the JavaScript version we need. But we can’t control the other dependencies (and their dependencies), which may or may not have Node.js specific requirements + dependencies.

I’ll be cheering you on. But here’s your first roadblock.

@msutkowski

You can specify a web worker output target. But you’re not wrong about various dependencies using non-web worker-friendly methods. An immediate example I can give you there is validation libraries on workers.

Wrangler won’t currently support a WASM build target, and workers don’t currently support TCP - sooooooooo, this is a hard blocker for anything Prisma related.

Deploy just Redwood-Worker-Functions to Cloudflare?

@thedavid

So maybe we have limitations. BUT it would still be rad to know what Workers are good at and what we could use them for. Imagine a Redwood App that takes advantage of a hybrid cloud infrastructure:

Cloudflare CDN + Networking (DNS)

Specific Redwood-Worker-Functions deployed to Cloudflare

Other parts of API deployed to AWS Fargate container for persistence

I don’t know why we’d do this. But it’s possible. ¯_(ツ)_/¯

I do think there’s a lot of value we could unlock in Cloudflare Key/Value and Durable Objects when it comes to performance + caching. Just don’t know what the ROI would be on trying to figure out how to piece it all together.

@peterp

I too very much doubt that Prisma+Cloudflare will happen. Those two products just seem so out of sync, why globally distribute your workers, but then connect to a single database.

I think caching is probably the immediate answer, probably on the API-level. Maybe you have an API that’s exposed via GraphQL, but that’s cached globally. Then… you don’t really need global workers, just global varnish.

Prior Art??

workers-graphql-server

It’s like they read my mind.

An Apollo GraphQL server, built with Cloudflare Workers. Try a demo by looking at a deployed GraphQL playground.

Why this rules: Cloudflare Workers is a serverless application platform for deploying your projects across Cloudflare’s massive distributed network. Deploying your GraphQL application to the edge is a huge opportunity to build consistent low-latency API servers, with the added benefits of “serverless” (I know, the project has server in it):

usage-based pricing

no cold starts

instant, easy-to-use deployment software, using Wrangler

By the way - as a full-stack developer who loves GraphQL, and the developer advocate for Cloudflare Workers, I would love to see what you build with this! Let me know on Twitter!

Usage

You can begin building your own Workers GraphQL server by installing Wrangler, the Workers command-line tool, and generating a new project:

wrangler generate my-graphql-server https://github.com/signalnerve/workers-graphql-server

You’ll need to configure your project’s wrangler.toml file to prepare your project for deployment. See the docs (“Configuring and Publishing”) for a guide on how to do this. Note that you’ll need to find your Cloudflare API keys to set up your config file. The source for this project includes an example external REST data source, and defined types for the PokeAPI, as an example of how to integrate external APIs.

To start using the project, configure your graphQLOptions object in src/index.js:

const graphQLOptions = {
  baseEndpoint: '/', // String
  playgroundEndpoint: '/___graphql', // ?String
  forwardUnmatchedRequestsToOrigin: false, // Boolean
  debug: false, // Boolean
  cors: true, // Boolean or Object to further configure
  kvCache: false // Boolean
}

Endpoints

Make requests to your GraphQL server at the baseEndpoint (e.g. graphql-on-workers.signalnerve.com/) and, if configured, try GraphQL queries at the playgroundEndpoint (e.g. graphql-on-workers.signalnerve.com/___graphql).

Worker GraphQL

GraphQL on Workers was inspired by this blog post, but using Apollo Server has a massive bundle size or can’t bundle due to dependency on graphql-upload (a node.js dependency). Using worker-graphql resulted in a build of < 50KB.

Installation

npm install @borderless/worker-graphql --save

Usage

import { processGraphQL } from "@borderless/worker-graphql";
import { makeExecutableSchema } from "graphql-tools";

const schema = makeExecutableSchema({
  typeDefs: `
    type Query {
      hello: String
    }
  `,
  resolvers: {
    Query: {
      hello: () => "Hello world!",
    },
  },
});

// Wrap `processGraphQL` with CORS support.
const handler = async (req: Request) => {
  if (req.method.toUpperCase() === "OPTIONS") {
    return new Response(null, {
      status: 204,
      headers: {
        "Access-Control-Allow-Methods": "GET,POST",
        "Access-Control-Allow-Headers":
          req.headers.get("Access-Control-Request-Headers") || "Content-Type",
        "Access-Control-Allow-Origin": "*",
      },
    });
  }

  const res = await processGraphQL(req, { schema });
  res.headers.set("Access-Control-Allow-Origin", "*");
  return res;
};

addEventListener("fetch", (event) => {
  event.respondWith(handler(event.request));
});

ajcwebdev · May 3, 2021, 3:16am

I don’t know the exact implications of this, since it’s only about two weeks old but it seems like potentially a big deal:

Node.js support in Cloudflare Workers

The first thing to note is that you may think the title is suggesting that Node.js is supported in Cloudflare Workers, but this is actually meant to give a summary of the current state of Node support today and where it will go in the future.

As David noted, this has not been very clear in the past, so I applaud the team for trying to better communicate the technical limitations of the current implementation.

Support today

It seems like anything using Webpack should in theory be usable. I rarely can get Webpack to work in the first place, so this doesn’t exactly scream killer DX to me.

For over 20k packages, Workers supports this magic already: any Node.js package that uses webpack or another polyfill bundler runs within our environment today.

But it should be at least a starting point, in theory, since Redwood was heavily built with Webpack, you would think there should be a lot of overlap. However, as some may know, Peter recently saw the light of esbuild and we shouldn’t be thinking of Redwood through the lens of Webpack for much longer.

We don’t need to speculate though, because the team put together a handy dandy Works on Workers website. But unless I’m missing something I don’t see 20k packages at that link, I see 32, so looks like it’s a bit of a work in progress still.

Use Gatsby on Cloudflare Workers - in theory

Once you select a package, you can use webpack to bundle it all up with the wrangler CLI and deploy onto Workers. They provide the following example of bundling dependencies in action with Gatsby.

Unfortunately supporting a static site generator is a much different use case from Redwood, so best case scenario is we could probably leverage this work with the web side but will still hit the same roadblocks with Prisma.

Okay, but seriously though, when is Node support coming to Workers?

We want to go even further and support the most important modules, even if they do rely on native code. Our approach will be to reimplement supported modules and polyfill package functionality directly into the Workers runtime. This doesn’t mean we’re shifting our runtime to run on Node.js. In fact, here are two important security and design reasons why we are not:

Node.js was not designed to be a sandbox, which was made apparent by their vm module that says “do not use it to run untrusted code.”

For proper sandboxing, Node.js would’ve forced us to build a container-based runtime that both doesn’t scale and isn’t as performant as Isolates. Without containers, we were able to design a system that has 0ms cold starts.

However, there are other ways we can be Node.js compatible without necessarily supporting the entire runtime. What’s up first? We’ll support Stripe.js SDK and Twilio Client JS SDK. We’ll also build support for the net module, so you can run popular database libraries.

But we want to hear from you! We created a leadership board for you to vote on which popular libraries/APIs matter the most.

So the answer is a very diplomatic never. Time for Denowood?

ajcwebdev · October 19, 2021, 6:46am

Deploying Prisma to Cloudflare Workers

The Prisma Deployment Guides now includes a section for Deploying to Cloudflare Workers. I ran through the guide and wasn’t able to make it quite to the end, so don’t expect to ship this to production next week.

But the issues are likely minor in the grand scheme of making this whole thing work in the first place, so watch this space for updates. I anticipate the final bugs will be ironed out in the near future and then this will be ready for developers to start testing.

ajcwebdev · October 21, 2021, 12:08am

PrismaWorkers is a go.

prisma-mongodb-cloudflare.anthonycampolo.workers.dev

ajcwebdev · November 15, 2021, 5:15pm

Big week of announcements for Cloudflare related to databases.

Workers adds support for two modern data platforms: MongoDB Atlas and Prisma

Today we’re excited to announce that, in addition to our existing partners Fauna and Macrometa, Cloudflare Workers has added support for Prisma and MongoDB Atlas.

Nothing new here for those in the know .

Introducing Relational Database Connectors

In April, we asked which Node libraries you wanted us to support, and four of the top five requests were related to databases. For this Full Stack Week, we asked ourselves: how could we support relational databases in a way that aligned with our design goals?

Today, we’re taking a first step towards that world by announcing support for relational databases, including Postgres and MySQL from Workers.

Making connections with TCP and Sockets for Workers

Out of the box, Cloudflare Workers support the ability to open HTTP and WebSocket connections using the standardized fetch and WebSocket APIs. With just a few internal changes to make it operational in Workers, we’ve developed an example using an off-the-shelf driver (in this example, a Deno-based Postgres client driver) to communicate with a remote Postgres server via WebSocket over a secure Cloudflare Tunnel.
import { Client } from './driver/postgres/postgres'

export default {
 async fetch(request: Request, env, ctx: ExecutionContext) {
   try {
     const client = new Client({
       user: 'postgres',
       database: 'postgres',
       hostname: 'https://db.example.com',
       password: '',
       port: 5432,
     })
     await client.connect()
     const result = await client.queryArray('SELECT * FROM users WHERE uuid=1;')
     ctx.waitUntil(client.end())
     return new Response(JSON.stringify(result.rows[0]))
   } catch (e) {
     return new Response((e as Error).message)
   }
 },
}

ajcwebdev · November 25, 2021, 8:20am

helix-flare is a GraphQL-Helix package that aims to simplify building GraphQL services with Cloudflare Workers and Durable Objects. Workers act as a proxy while delegating execution to Durable Objects.

import helixFlare from 'helix-flare'
import { makeExecutableSchema } from '@graphql-tools/schema'

export default {
  async fetch(request: Request) {
    const typeDefs = `type Query { hello: String! }`
    const schema = makeExecutableSchema({
      typeDefs,
      resolvers: { Query: { user: () => 'Hello World 🌍' }, },
    })

    return helixFlare(request, schema)
  },
}

fetch('https://my.worker.dev/graphql', {
  body: JSON.stringify({ query: '{ hello }' }),
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
})

tctrautman · February 9, 2022, 2:53am

@ajcwebdev are you aware of any major relevant updates on this since November?

ajcwebdev · February 9, 2022, 4:02am

Hey @tctrautman, thanks for the ping! This has remained totally theoretical throughout the last year because making it work has required Cloudflare to become more Node friendly and Redwood to become more Cloudflare friendly.

But at this point, as far as I can tell, every major dependency of a vanilla Redwood app is capable of running on a combination of Cloudflare Pages for the web side and Cloudflare Workers for the api side (connected to an external database hosted elsewhere).

But there’s a wide variety of other dependencies used throughout Redwood apps in the wild, and even among dependencies used in the docs there will be a large divergence between Worker compatible and non-compatible libraries. All of the authentication plugins, for example, would need a thorough vetting.

With that said, at this point it’s really just a matter of having someone with enough Cloudflare knowledge and Redwood knowledge to attempt it and see where the edge cases arise. Run yarn create redwood-app redwood-cloudflare, add wrangler, and see what happens.

tctrautman · February 9, 2022, 4:13am

Thank you for summarizing! This is super helpful and much appreciated