The other day I was getting a 502 error when doing a complex query to my database. I knew the problem, my query was running longer than 10 seconds. With Netlify’s functions being a wrapper around aws lambda calls that’s a limitation of using Netlify.
To solve this I decided I’d move from Netlify + Railway to Render.
Moving to Render from Netlify + Railway
I wanted to write this down so others may find it useful. A few thing about my site.
- I was using some headers to be a good steward re: security. https://observatory.mozilla.org/
- I am using dbAuth
- I have a bunch of data I needed to migrate.
Lets get into it.
Table of Contents
- Create a branch and setup the render deploy
- Deploy to render
- Connect the Web Side to the API Side
- Moving the data
- Setting your environment variables
- Logging in
- Configuring your custom domain
- Adding Custom Headers
- Addressing any custom routes
- Adding Redis to the mix
Create a branch
First I created a branch on my repo called “move-to-render” but this will let you make the changes without breaking the current site.
git checkout -b move-to-render
Then we need to setup the render deploy
yarn rw setup deploy render
This creates a render.yaml file which we will need to make some changes to after we have render make our services initially.
Add the changes to your new branch and push them up.
git add .
git commit -m "added render.yaml to deploy to render"
git push
Deploy to Render.com
Now we’re going to follow these steps (I’m summarizing below): Deploy RedwoodJS | Render · Cloud Hosting for Developers
- create/login to your account on render.com
- From your dashboard, click New, then blueprint, It’ll ask for your repository, select it, then choose the branch you defined above, “deploy-to-render”
- Take a 5 minute break to allow it all to build.
Connect the Web Side to the API Side
Now we need to wire up the front end to the web services, to do this we need to change the render.yaml file. First we need the URL for the API services.
- Goto your dashboard
- find the web service you just deployed, click it
- Note the URL for the service on that page, copy it.
- Open render.yaml and find the
source: /.redwood/function/* - Update the next line (should be destination) value to your api endpoint followed by
/* - It should look like
destination: https://whateverYourApiUrlIs.onrender.com/* - Save that. and push it up.
git add .
git commit -m "updated rewrite rules"
git push
Moving the data
This is a more complex step so I’m going to break it down into steps.
Download DBGate
I use a tool called dbgate to connect, extract and moving my data. You can use any tool you’d like but I found this worked for me.
Once downloaded lets open it up.
Connect to your existing database
We’re going to add a connection here.
- Click Add a new connection.
- Choose Postgres
- Choose use Database URL
- Login to netlify and copy your
DATABASE_URLvalue - Name this “Railway Database”
Connect to your new database on Render
- Add a new connection to your new database
- Choose Postgres
- Choose use databse url
- log in to render.com and goto your the database
- copy the External Database URL.
- Paste that in the database url
- Append
?ssl=trueto the database url. - Name it something like “Render Database”
Move the data via Import Wizard
Now that you have both databases lets do the import. Before we get started you may need to move your tables in a hierarchy order. I mean move the tables with no dependencies and add the ones with dependencies after they are made.
- Right click on the render database in the top left and click Import wizard
- This loads up a new interface.
- On the left
- Storage Type: Database
- Server: Netlify Database
- Database: postgres
- Tables, select all the tables you name in your schema.prisma file and _prisma_migrations
- On the right
- Storage Type: Database
- Server: Render Database
- Database: postgres
- Now your data may have dependencies, and in those cases you’ll need to import in stages. e.g. if you have a users table and a group member table and a group table. You can probably import users and groups but you’ll need to import group member AFTER users and groups is moved. With that caveat it took me some time to get this to move over with out errors.
- Start running these imports until you get them all.
Fixing your autoincrement columns
My autoincrement fell out of sync for two of my tables. If you’re getting an error later when you’re creating records I had to run the following sql against each table.
SELECT setval(pg_get_serial_sequence('"ItemView"', 'id'), coalesce(max(id)+1, 1), false) FROM "ItemView";
You’ll need to replace the table (ItemView) and id field appropriately but that fixed me all up.
Setting your environment variables
Let’s set up our environment variables. With DBAuth you need to set a session secret, you can reuse your session secret from your past host, OR you can generate a new one. I reused my old one but generating a new one would mean also resetting your password to authenticate. To set the environment variable on render goto the API service, on the left is a environment link. Add a new one wiht the name SESSION_SECRET and the value from your terminal or your previous host.
yarn rw g secret
Logging in
At this point you should be able to either log in or reset your password. Try it.
If you’re using a new secret, you’ll need to get your reset token and reset it.
You can connect locally by connecting to prisma;
yarn rw prisma studio
navigate to your user table, find your account, copy the reset token and goto your sites reset password link: /reset-password?resetToken=PASTEYOURTOKENHERE
Configuring your custom domain
I am using my own domain. I assume you’ll want to too.
In Render this was really simple.
- Goto the Static Site and click settings.
- scroll down until you get to the custom domains
- add your domain
- follow instructions and update your dns to match the new domain
In my case I was taking the old domain. So I had to delete the ipv4 and ipv6 versions on netlify’s domain management. This will likely be similiar for you but may be different depending how you manage your DNS entries.
Adding Custom Headers
Let’s add on those headers missing from before.
Back in the render.yaml file find the below the routes you midified earlier is a big comment block, I removed this here. Than I added a new section called headers at the same level;
You’ll need to add one for each header you want.
Addressing any custom routes
I had a custom function to build a external RSS feed I forgot about. To add that so /index.xml rendered the equivalent of /.redwood/functions/rss I just needed to add another route to my render.yaml file.
routes:
- type: rewrite
source: /index.xml
destination: https://news-tskr-io-api.onrender.com/rss
Congratulations
Edit 1 2022-11-10 Adding Redis to the mix!
So I also had Redis set up at one point and I wanted to set this back up. I’m still learning how to deal with caching but I wanted to share how to set this up on Render in 4 Steps.
Step one, update render.yaml
In your render.yaml file you’ll want to add this as another service;
- type: redis
name: cache
#ipAllowList: [] # only allow internal connections
plan: free # optional (defaults to starter)
maxmemoryPolicy: allkeys-lfu # optional (defaults to allkeys-lru). Rails recommends allkeys-lfu as a default.
ipAllowList:
- source: 0.0.0.0/0 #I wasn't able to get this work using just the internal connection
description: public
That will add another item to your dashboard’s blueprint for your service.
Step two, add the envelop packages
Next you’ll need to add the packages for redis;
yarn workspace api add @envelop/response-cache
yarn workspace api add @envelop/response-cache-redis
Step three, configure graphql.js
Next you need to configure graphql to use it. Open ./api/src/functions/graphql.js and make these changes
// Add these two lines below
import { useResponseCache } from '@envelop/response-cache'
import { createRedisCache } from '@envelop/response-cache-redis'
import { createGraphQLHandler } from '@redwoodjs/graphql-server'
import directives from 'src/directives/**/*.{js,ts}'
import sdls from 'src/graphql/**/*.sdl.{js,ts}'
import services from 'src/services/**/*.{js,ts}'
import { getCurrentUser } from 'src/lib/auth'
import { db } from 'src/lib/db'
import { logger } from 'src/lib/logger'
import { redis } from 'src/lib/redis' // this is needed still but we'll do this next
export const handler = createGraphQLHandler({
loggerConfig: { logger, options: {} },
getCurrentUser,
directives,
sdls,
services,
extraPlugins: [
// this here needs to be added for the redis cache
useResponseCache({
cache: createRedisCache({ redis }),
ttl: 0, // 1 minute
includeExtensionMetadata: true,
ttlPerSchemaCoordinate: {
// there's more ways to control this but here's my first stab at it
'Query.PublicFeeds': 1000 * 60 * 5, // 5 minutes in ms
'Query.searchPublicItems': 1000 * 60 * 5, // 5 minute in ms
},
}),
],
onException: () => {
// Disconnect from your database with an unhandled exception.
db.$disconnect()
},
})
Step four, create redis.js
You’ll need to create a ./api/src/lib/redis.js file. I had this using the redis url before but I couldn’t get that to work on Render. Here’s my file’s content;
import Redis from 'ioredis'
//export const redis = new Redis(process.env.REDIS)
export const redis = new Redis({
username: process.env.REDIS_SERVICE_NAME, // Render Redis name, red-xxxxxxxxxxxxxxxxxxxx
host: process.env.REDIS_HOST, // Render Redis hostname, REGION-redis.render.com
password: process.env.REDIS_PASSWORD, // Provided password
port: process.env.REDIS_PORT || 6379, // Connection port
tls: true, // TLS required when externally connecting to Render Redis
})
Step five, deploy to render, and set the environment variables
Now you’ll need to do a do a push to have Render build you’re redis cache, but when you do you will be able to capture the service, host, password from the External Redis URL.
Click on the External Redis URL and we’ll tear it down.
rediss://serviceName:password@redisHost.render.com:6379So if this were my string I’d make environment variables like such (locally) and also on render’s API service;
REDIS_HOST=redisHost.render.com
REDIS_PASSWORD=password
REDIS_SERVICE_NAME=serviceName
You may need to do another deploy here to trigger the builds with your variables.
