Deployment DB Migrations

Get Help and Help Others
1

Hello - we are hosting on Azure, which I realize is not an officially supported deployment target. Using the guidelines written here by @talk2MeGooseman (and on his blog - Deploying RedwoodJS Server to Azure Web App Service) as a starting point (and with some updating), I have a deployment setup that works nicely on Azure with three separate instances - a static web app instance for the front-end, an app service for the API, and a Postgres flexible DB server. I’ve tied it into Azure CI/CD Pipelines, and it all works quite nicely - approve a PR to a watched branch, and if it builds successfully and is approved, it will auto-deploy to our test environment (not to prod, of course).

One question - none of the posts I’ve read specifically include how to do DB migrations. I have a nice Kudu deploy.sh script that gets run during deployment, which handles running yarn install and yarn rw build api.

I’ve looked over the docs and the RW source, in particular Data Migrations | RedwoodJS Docs . Since the migrations are already generated on our dev instances and in git, can I just run:

yarn rw data-migrate up

in our deploy.sh after it finishes building the API? Do I also need to run prisma migrate deploy (as referenced on New Vercel Deployment Build Process - #2 by viperfx )?

2

You should be fine on Azure!

For custom implementations, it’s important you understand the deployment steps required:

  1. build
  2. DB schema migration (prisma)
  3. data migration (dataMigrate)

There are CLI commands for each step.

Note: Our docs aren’t clear on this and it need to be updated — if you are using prerender with data hooks, you need to run that after step 3 above. Otherwise your data request for prerender will be out of sync with DB schema.

Limited overview of this process here in the docs:

3

Thanks! The clarification of schema vs actual data migration was helpful. Until we setup our test environment on Azure, we’d been doing local dev and then a dev deployment on a local Linux box using baremetal deployment.

I just added a new model and removed an existing field and generated two separate migrations. Modified our deploy.sh and just pushed everything up to Azure Dev Ops, and the build/deploy ran perfectly - the DB migrations were successfully applied and I’ve verified directly on our Postgres box.

I added two commands after yarn rw build api so effectively, it does these three in sequence:

yarn rw build api
yarn rw prisma migrate deploy
yarn rw data-migrate up

For anyone who is interested and using Azure, this is our (current) deploy.sh that is run on our API server CI/CD pipeline (deployment processes are different for web vs API). Note that due to what seems to be a bug running selectNodeVersion() (it just fails on Azure, at least in a default configured Node.js React app server instance and I didn’t want to spend much time figuring out the fix) so the script hardcodes the path to npm from nodejs 18.16.0. I’ve also added additional debugging output so I can better follow each step

#!/bin/bash

# ----------------------
# KUDU Deployment Script
# Version: 1.0.17
# ----------------------

# Helpers
# -------

exitWithMessageOnError () {
  if [ ! $? -eq 0 ]; then
    echo "An error has occurred during web site deployment."
    echo $1
    exit 1
  fi
}

# Prerequisites
# -------------

# Verify node.js installed
echo About to verify if node.js is installed
hash node 2>/dev/null
node --version
exitWithMessageOnError "Missing node.js executable, please install node.js, if already installed make sure it can be reached from current environment."
echo Verified node.js is installed

# Setup
# -----

SCRIPT_DIR="${BASH_SOURCE[0]%\\*}"
SCRIPT_DIR="${SCRIPT_DIR%/*}"
ARTIFACTS=$SCRIPT_DIR/../artifacts
KUDU_SYNC_CMD=${KUDU_SYNC_CMD//\"}

if [[ ! -n "$DEPLOYMENT_SOURCE" ]]; then
  DEPLOYMENT_SOURCE=$SCRIPT_DIR
fi

if [[ ! -n "$NEXT_MANIFEST_PATH" ]]; then
  NEXT_MANIFEST_PATH=$ARTIFACTS/manifest

  if [[ ! -n "$PREVIOUS_MANIFEST_PATH" ]]; then
    PREVIOUS_MANIFEST_PATH=$NEXT_MANIFEST_PATH
  fi
fi

if [[ ! -n "$DEPLOYMENT_TARGET" ]]; then
  DEPLOYMENT_TARGET=$ARTIFACTS/wwwroot
else
  KUDU_SERVICE=true
fi

if [[ ! -n "$KUDU_SYNC_CMD" ]]; then
  # Install kudu sync
  echo Installing Kudu Sync
  npm install kudusync -g --silent
  exitWithMessageOnError "npm failed"

  if [[ ! -n "$KUDU_SERVICE" ]]; then
    # In case we are running locally this is the correct location of kuduSync
    KUDU_SYNC_CMD=kuduSync
  else
    # In case we are running on kudu service this is the correct location of kuduSync
    KUDU_SYNC_CMD=$APPDATA/npm/node_modules/kuduSync/bin/kuduSync
  fi
fi

# Node Helpers
# ------------

selectNodeVersion () {
  if [[ -n "$KUDU_SELECT_NODE_VERSION_CMD" ]]; then
    SELECT_NODE_VERSION="$KUDU_SELECT_NODE_VERSION_CMD \"$DEPLOYMENT_SOURCE\" \"$DEPLOYMENT_TARGET\" \"$DEPLOYMENT_TEMP\""
    eval $SELECT_NODE_VERSION
    exitWithMessageOnError "select node version failed"

    if [[ -e "$DEPLOYMENT_TEMP/__nodeVersion.tmp" ]]; then
      NODE_EXE=`cat "$DEPLOYMENT_TEMP/__nodeVersion.tmp"`
      exitWithMessageOnError "getting node version failed"
    fi

    if [[ -e "$DEPLOYMENT_TEMP/__npmVersion.tmp" ]]; then
      NPM_JS_PATH=`cat "$DEPLOYMENT_TEMP/__npmVersion.tmp"`
      exitWithMessageOnError "getting npm version failed"
    fi

    if [[ ! -n "$NODE_EXE" ]]; then
      NODE_EXE=node
    fi

    NPM_CMD="\"$NODE_EXE\" \"$NPM_JS_PATH\""
  else
    NPM_CMD=npm
    NODE_EXE=node
  fi
}

##################################################################################################################################
# Deployment
# ----------

echo Handling node.js deployment.

# 1. KuduSync
if [[ "$IN_PLACE_DEPLOYMENT" -ne "1" ]]; then
  "$KUDU_SYNC_CMD" -v 50 -f "$DEPLOYMENT_SOURCE" -t "$DEPLOYMENT_TARGET" -n "$NEXT_MANIFEST_PATH" -p "$PREVIOUS_MANIFEST_PATH" -i ".git;.hg;.deployment;deploy.sh"
  exitWithMessageOnError "Kudu Sync failed"
fi

echo "Kudu Sync succeeded. Selecting node version"

# 2. Select node version
echo "Forcing node version"
NPM_CMD="node /opt/nodejs/18.16.0/bin/npm"
# selectNodeVersion

echo "Node version selected, installing npm packages."

# 3. Install npm packages
if [ -e "$DEPLOYMENT_TARGET/package.json" ]; then
  cd "$DEPLOYMENT_TARGET"
  echo "Running yarn install"
  eval yarn install
  echo "Building API"
  eval yarn rw build api
  exitWithMessageOnError "yarn rw build api failed"
  # DB schema migrations
  echo "Running DB schema migrations..."
  eval yarn rw prisma migrate deploy
  exitWithMessageOnError "Prisma migrations failed"
  echo "Running data migrations..."
  eval yarn rw data-migrate up
  exitWithMessageOnError "Data migrations failed"
  cd - > /dev/null
fi

##################################################################################################################################
echo "Finished successfully."

I’ve been documenting everything as I’ve built out the environment. I will verify when I go to build out our prod environment, and will be happy to share what I’ve written at that point.

2 Likes
4

Please do! This would be very welcome.