What is the procedure to run the new docker experimental functionality?

Hello,

I’m using the last RedwoodJS version (6.6.0) and I’ve realized there’s a new experimental Docker feature in it… Let’s say I create a new project from scratch with:

yarn create redwood-app myproject

After entering its folder I’ve executed yarn rw exp setup-docker and then I tried running the development docker-compose file with:

docker-compose -f docker-compose.dev.yml up --build

I’ve received the following error then:

docker-compose -f docker-compose.dev.yml up -d --build

Creating volume "myproject_node_modules" with default driver
Creating volume myproject_postgres" with default driver
Building redwood
[+] Building 0.7s (10/19)                                                                                                                                                                                                                                         docker:default
 => [internal] load .dockerignore                                                                                                                                                                                                                                           0.0s
 => => transferring context: 182B                                                                                                                                                                                                                                           0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                        0.0s
 => => transferring dockerfile: 3.93kB                                                                                                                                                                                                                                      0.0s
 => [internal] load metadata for docker.io/library/node:18-bookworm-slim                                                                                                                                                                                                    0.7s
 => [internal] load build context                                                                                                                                                                                                                                           0.0s
 => => transferring context: 314B                                                                                                                                                                                                                                           0.0s
 => CACHED [internal] settings cache mount permissions                                                                                                                                                                                                                      0.0s
 => CANCELED [base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108f0c624b783c37                                                                                                                              0.0s
 => => resolve docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108f0c624b783c37                                                                                                                                              0.0s
 => => sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108f0c624b783c37 1.21kB / 1.21kB                                                                                                                                                                              0.0s
 => CACHED [base  2/14] RUN apt-get update && apt-get install -y     openssl     && rm -rf /var/lib/apt/lists/*                                                                                                                                                             0.0s
 => CACHED [base  3/14] WORKDIR /home/node/app                                                                                                                                                                                                                              0.0s
 => ERROR [base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                                                                                                                                                                   0.0s
 => ERROR [base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                                                                                                                                                                                 0.0s
------
 > [base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins:
------
------
 > [base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
Dockerfile:14
--------------------
  12 |
  13 |     COPY --chown=node:node .yarn/plugins .yarn/plugins
  14 | >>> COPY --chown=node:node .yarn/releases .yarn/releases
  15 |     COPY --chown=node:node .yarnrc.yml .
  16 |     COPY --chown=node:node package.json .
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 058ab57a-1371-40e9-84bd-1ed0c9849638::65465ib7ayxhtvo9su4nsajzl: "/.yarn/releases": not found
ERROR: Service 'redwood' failed to build : Build failed

Am I missing any step here? I’ve looked for a tutorial page on the RedwoodJS documentation and here in the forum but I didn’t find it.

Apparently it’s searching the files .yarn/plugins and .yarn/releases, but if I enter my .yarn folder I only see one file called install-state.gz

1 Like

Hey @ramuyk, I think you’ve just alerted me to a bug in yarn rw exp setup-docker as of v6.6.0 which just went out today (yesterday now). In that version, we upgraded new projects to yarn v4. That meant two things both related to the .yarn directory:

  • we don’t need .yarn/plugins anymore because they’re included in yarn v4
  • we don’t need .yarn/releases because yarn now recommends using corepack

I didn’t update yarn rw exp setup-docker to account for these changes, so it’s trying to copy files that don’t exist. I’ll work on that in the next patch.

For now you should be able to get things back to working by downgrading the version of yarn to v3.7.0, then running the experimental setup command again:

# If you're on yarn v4 right now and need to downgrade:
yarn set version v3 --yarn-path
yarn
yarn rw exp setup-docker

Hi @dom

I’ve downgraded the yarn as you said:

yarn --version
3.7.0

yarn rw --version
6.6.0

I still see this error though:

docker-compose -f docker-compose.dev.yml up --build
Building redwood
[+] Building 0.7s (10/19)                                                                                               docker:default
 => [internal] load .dockerignore                                                                                                 0.0s
 => => transferring context: 182B                                                                                                 0.0s
 => [internal] load build definition from Dockerfile                                                                              0.0s
 => => transferring dockerfile: 3.93kB                                                                                            0.0s
 => [internal] load metadata for docker.io/library/node:18-bookworm-slim                                                          0.6s
 => [internal] load build context                                                                                                 0.0s
 => => transferring context: 484B                                                                                                 0.0s
 => CANCELED [base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea10  0.0s
 => => resolve docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108f0c624b783c37    0.0s
 => CACHED [internal] settings cache mount permissions                                                                            0.0s
 => CACHED [base  2/14] RUN apt-get update && apt-get install -y     openssl     && rm -rf /var/lib/apt/lists/*                   0.0s
 => CACHED [base  3/14] WORKDIR /home/node/app                                                                                    0.0s
 => CACHED [base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                        0.0s
 => ERROR [base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                                       0.0s
------
 > [base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
Dockerfile:14
--------------------
  12 |
  13 |     COPY --chown=node:node .yarn/plugins .yarn/plugins
  14 | >>> COPY --chown=node:node .yarn/releases .yarn/releases
  15 |     COPY --chown=node:node .yarnrc.yml .
  16 |     COPY --chown=node:node package.json .
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 058ab57a-1371-40e9-84bd-1ed0c9849638::jb5l5fa6ih5lns5dmq58ih21n: "/.yarn/releases": not found
ERROR: Service 'redwood' failed to build : Build failed

It’s still looking for those files. This is a project I’ve started from scratch again… Commenting these lines in the Dockerfile doesn’t solve the issue either. Is there anything else I should do? Besides the yarn and RedwoodJS version, my docker is the 24.0.7.

Hmmm I’m guessing you didn’t re-run the docker setup command? I included it in the codeblock because it’d be necessary to re-run afterwards to pull in yarn’s workspace-tools plugin:

But if you don’t want to do that, you can add workspace tools:

yarn plugin import workspace-tools

That’ll add the .yarn/plugins directory.

Actuallly, I ran everything from scratch. First

yarn create redwood-app myproject

Then

cd myproject
yarn rw exp setup-docker
docker compose -f docker-compose.dev.yml up

[+] Building 0.7s (15/36)                                                                                               docker:default
 => [redwood internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 182B                                                                                                 0.0s
 => [redwood internal] load build definition from Dockerfile                                                                      0.0s
 => => transferring dockerfile: 3.93kB                                                                                            0.0s
 => [console internal] load build definition from Dockerfile                                                                      0.0s
 => => transferring dockerfile: 3.93kB                                                                                            0.0s
 => [console internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 182B                                                                                                 0.0s
 => [redwood internal] load metadata for docker.io/library/node:18-bookworm-slim                                                  0.6s
 => [redwood internal] load build context                                                                                         0.0s
 => => transferring context: 801.20kB                                                                                             0.0s
 => [console base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108  0.0s
 => CACHED [console internal] settings cache mount permissions                                                                    0.0s
 => [console internal] load build context                                                                                         0.0s
 => => transferring context: 824.59kB                                                                                             0.0s
 => CACHED [console base  2/14] RUN apt-get update && apt-get install -y     openssl     && rm -rf /var/lib/apt/lists/*           0.0s
 => CACHED [console base  3/14] WORKDIR /home/node/app                                                                            0.0s
 => CACHED [redwood base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                0.0s
 => ERROR [redwood base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                               0.0s
 => CACHED [console base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                0.0s
 => ERROR [console base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                               0.0s
------
 > [redwood base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
------
 > [console base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
failed to solve: failed to compute cache key: failed to calculate checksum of ref 058ab57a-1371-40e9-84bd-1ed0c9849638::vw9gmygpz1cfgphka6a2xaexn: "/.yarn/releases": not found

If I try running your last command:

 yarn plugin import workspace-tools
➤ YN0000: Downloading https://github.com/yarnpkg/berry/raw/@yarnpkg/cli/3.7.0/packages/plugin-workspace-tools/bin/%40yarnpkg/plugin-workspace-tools.js
➤ YN0000: Saving the new plugin in .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
➤ YN0000: Done in 7s 33ms

And then building it:

 docker compose -f docker-compose.dev.yml up

[+] Building 11.3s (15/36)                                                                                              docker:default
 => [redwood internal] load build definition from Dockerfile                                                                      0.0s
 => => transferring dockerfile: 3.93kB                                                                                            0.0s
 => [redwood internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 182B                                                                                                 0.0s
 => [console internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 182B                                                                                                 0.0s
 => [console internal] load build definition from Dockerfile                                                                      0.0s
 => => transferring dockerfile: 3.93kB                                                                                            0.0s
 => [console internal] load metadata for docker.io/library/node:18-bookworm-slim                                                 11.3s
 => [redwood internal] load build context                                                                                         0.0s
 => => transferring context: 52.06kB                                                                                              0.0s
 => CACHED [console internal] settings cache mount permissions                                                                    0.0s
 => [console base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108  0.0s
 => [console internal] load build context                                                                                         0.0s
 => => transferring context: 75.45kB                                                                                              0.0s
 => CACHED [console base  2/14] RUN apt-get update && apt-get install -y     openssl     && rm -rf /var/lib/apt/lists/*           0.0s
 => CACHED [console base  3/14] WORKDIR /home/node/app                                                                            0.0s
 => CACHED [redwood base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                0.0s
 => ERROR [redwood base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                               0.0s
 => CACHED [console base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                0.0s
 => ERROR [console base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                               0.0s
------
 > [redwood base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
------
 > [console base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases:
------
failed to solve: failed to compute cache key: failed to calculate checksum of ref 058ab57a-1371-40e9-84bd-1ed0c9849638::31a1xw2matd0tx9xctuc31x7w: "/.yarn/releases": not found

Even though the plugins directory is there now, the error is only in .yarn/releases… It doesn’t exist inside my .yarn folder.

Could you watch what I’m doing here and tell me if you’re doing anything differently?

1 Like

Cool, my mistake was that I thought that the global version of yarn would remain the same inside the newly created project… Changing the yarn version after creating the project worked, thanks!

1 Like

PR here for updating the Dockerfile

1 Like

I’m not sure if this is the right place or if I should open a new issue, but I have an already existing RedwoodJS project on version 6.3.1… I’ve failed to make the docker experimental feature work after an upgrade… Basically, I’ve upgraded my project:

yarn rw upgrade

After that, I have the following versions in my project:

yarn --version
3.6.3

yarn rw --version
6.6.0

Then I’ve added the docker setup:

yarn rw exp setup-docker
✔ Confirmation
✔ Adding the official yarn workspace-tools plugin...
✔ Adding @redwoodjs/api-server and @redwoodjs/web-server...
✔ Successfully wrote file `./.dockerignore`
✔ Adding postgres to .gitignore...
✔ Adding config to redwood.toml...

We've written four files:

- ./Dockerfile
- ./.dockerignore
- ./docker-compose.dev.yml
- ./docker-compose.prod.yml

To start the docker compose dev:

  docker compose -f docker-compose.dev.yml up

Then, connect to the container and migrate your database:

  docker compose -f ./docker-compose.dev.yml run --rm -it console /bin/bash
  root@...:/home/node/app# yarn rw prisma migrate dev

Lastly, ensure you have Docker. If you don't, see https://docs.docker.com/desktop/

There's a lot in the Dockerfile and there's a reason for every line.
Be sure to check out the docs: https://redwoodjs.com/docs/docker

Now if I run the command for starting the container it stops the build on the 11th step:

docker compose -f docker-compose.dev.yml up
[+] Building 8.3s (18/36)                                                                                                docker:default
 => [redwood internal] load .dockerignore                                                                                          0.0s
 => => transferring context: 182B                                                                                                  0.0s
 => [redwood internal] load build definition from Dockerfile                                                                       0.0s
 => => transferring dockerfile: 3.93kB                                                                                             0.0s
 => [redwood internal] load metadata for docker.io/library/node:18-bookworm-slim                                                   1.3s
 => [console internal] load .dockerignore                                                                                          0.0s
 => => transferring context: 182B                                                                                                  0.0s
 => [console internal] load build definition from Dockerfile                                                                       0.0s
 => => transferring dockerfile: 3.93kB                                                                                             0.0s
 => [redwood internal] load build context                                                                                          0.1s
 => => transferring context: 3.27MB                                                                                                0.0s
 => CACHED [redwood internal] settings cache mount permissions                                                                     0.0s
 => [redwood base  1/14] FROM docker.io/library/node:18-bookworm-slim@sha256:fe687021c06383a2bc5eafa6db29b627ed28a55f6bdfbcea108f  0.0s
 => [console internal] load build context                                                                                          0.7s
 => => transferring context: 149.66MB                                                                                              0.7s
 => CACHED [redwood base  2/14] RUN apt-get update && apt-get install -y     openssl     && rm -rf /var/lib/apt/lists/*            0.0s
 => CACHED [redwood base  3/14] WORKDIR /home/node/app                                                                             0.0s
 => CACHED [redwood base  4/14] COPY --chown=node:node .yarn/plugins .yarn/plugins                                                 0.0s
 => CACHED [redwood base  5/14] COPY --chown=node:node .yarn/releases .yarn/releases                                               0.0s
 => CACHED [redwood base  6/14] COPY --chown=node:node .yarnrc.yml .                                                               0.0s
 => CACHED [redwood base  7/14] COPY --chown=node:node package.json .                                                              0.0s
 => CACHED [redwood base  8/14] COPY --chown=node:node api/package.json api/                                                       0.0s
 => CACHED [redwood base  9/14] COPY --chown=node:node web/package.json web/                                                       0.0s
 => [redwood base 10/14] COPY --chown=node:node yarn.lock .                                                                        0.1s
 => [redwood base 11/14] RUN --mount=type=cache,target=/home/node/.yarn/berry/cache,uid=1000     --mount=type=cache,target=/home/  6.8s
 => => # ➤ YN0060: │ web@workspace:web provides react-dom (p581c4) with version 18.2.0, which doesn't satisfy what @redwoodjs/router an
 => => # d some of its descendants request
 => => # ➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements <hash> for details, where <hash> is
 => => #  the six-letter p-prefixed code
 => => # ➤ YN0000: └ Completed in 0s 363ms
 => => # ➤ YN0000: ┌ Fetch step

If I try to create a project from scratch as in my original question it works fine, I’d like to avoid manually passing my code to a new project to solve this problem. Does anyone have any idea of what could be happening?

I’m not sure how the output of yarn explain peer-requirements could help me, it generates more than 1000 lines of output… However, even in a functional project that I’ve started from scratch, this command still returns an output with hundreds of lines.

Referring to Docker experts: @dom and ans @Josh-Walker-GM

Hey @ramuyk, just to clarify, the issue is just that yarn is printing a lot? I.e. the containers actually start up just fine?

Hi @dom

Not exactly, it freezes on the 11th step while building the Dockerfile

=> [redwood base 11/14] RUN --mount=type=cache,target=/home/node/.yarn/berry/cache,uid=1000 --mount=type=cache,target=/home/ 6.8s
=> => # ➤ YN0060: │ web@workspace:web provides react-dom (p581c4) with version 18.2.0, which doesn’t satisfy what @redwoodjs/router an
=> => # d some of its descendants request
=> => # ➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements for details, where is
=> => # the six-letter p-prefixed code
=> => # ➤ YN0000: └ Completed in 0s 363ms
=> => # ➤ YN0000: ┌ Fetch step

In this example, I had just opened it, but it stops there forever and the container is not running yet. It stays hours in it and never passes this step (this is in a project I’ve upgraded from version 6.3.1).

Ah, I’ve executed yarn cache clean and now it works… Even if I try to clone my repository from scratch and follow the procedure I’ve described in my last question it works…

I think it was more a issue with my yarn installation than the RedwoodJS.

1 Like