Error deploying to Netlify: Cannot find module '../../../.redwood/datamodel.json'

I was able to get a bit further by changing the deploy command in the netlify. toml file but still getting this weird issue at the end

I changed from:

command = "yarn rw deploy netlify"

to:

command = "yarn rw record init && yarn rw deploy netlify"

And not I get to the end of the deploy and it says that some how it was canceled but I didn’t cancel it.

cc: @rob since it relates to RedwoodRecord

11:31:37 AM: ────────────────────────────────────────────────────────────────
11:31:37 AM:   2. Functions bundling                                         
11:31:37 AM: ────────────────────────────────────────────────────────────────
11:31:37 AM: ​
11:31:37 AM: Packaging Functions from api/dist/functions directory:
11:31:37 AM:  - auth.js
11:31:37 AM:  - graphql.js
11:31:37 AM: ​
11:32:05 AM: ​
11:32:05 AM: (Functions bundling completed in 28.2s)
11:32:05 AM: ​
11:32:05 AM: ────────────────────────────────────────────────────────────────
11:32:05 AM:   3. Deploy site                                                
11:32:05 AM: ────────────────────────────────────────────────────────────────
11:32:05 AM: ​
11:32:05 AM: Starting to deploy site from 'web/dist'
11:32:10 AM: Creating deploy tree 
11:32:10 AM: Creating deploy upload records
11:32:10 AM: 1 new files to upload
11:32:10 AM: 1 new functions to upload
11:32:16 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:19 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:22 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:24 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:27 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:30 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:36 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:43 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:32:54 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:33:10 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:33:37 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:33:53 AM: Request must be smaller than 69905067 bytes for the CreateFunction operation
11:33:53 AM: Failed to upload file: graphql
11:33:53 AM: Site deploy was successfully initiated
11:33:53 AM: ​
11:33:53 AM: (Deploy site completed in 1m 47.8s)
11:33:53 AM: ​
11:33:53 AM: ────────────────────────────────────────────────────────────────
11:33:53 AM:   Netlify Build Complete                                        
11:33:53 AM: ────────────────────────────────────────────────────────────────
11:33:53 AM: ​
11:33:53 AM: (Netlify Build completed in 3m 34.1s)
11:33:53 AM: Execution cancelled
11:33:53 AM: Error running command: Command was cancelled
11:33:53 AM: Failing build: Failed to build site
11:33:53 AM: Finished processing build request in 5m40.584157527s

Hmmmmmmm that error looks like the resulting function is too big for Netlify to be able to deploy to a Lambda: Request must be smaller than 69905067 bytes for the CreateFunction operation - Support - Netlify Support Forums So it’s Netlify cancelling the build. The more dependencies you have the bigger that function is going to get, but RR doesn’t have any wildly different deps than anything else (mostly just Prisma).

Unless RR itself is causing something to balloon in size…can you try running yarn rw build locally and see if that completes? I don’t think that prepares the built files in the same way that Netlify does, they have an extra step called zip-it-and-ship-it that prepares the files for AWS Lambda: https://github.com/netlify/zip-it-and-ship-it There may be a way to run that locally in Redwood…I’m waiting to hear back from some Redwood folks if that’s the case.

Hi, I tried that and it compiled quickly.

I made the repo I am working with public for the time being in case it helps having access to the files to see what’s going on.

Here is the link to the repo: https://github.com/aguscha333/agch

I confirmed that our build process with yarn rw build doesn’t use the zip-it-and-ship-it in any way. But, we could install and run it manually: https://github.com/netlify/zip-it-and-ship-it

It sounds like srcFolders just needs to be the api/src/functions folder, after going through yarn rw build. But that process might just blow up with the same error. :frowning:

We’ve actually got a doc on how to do it locally! Docs - Builds : RedwoodJS Docs

I ran those steps and it finished with no errors in 18s. Not sure what else to try

Can you find the zipped up files somewhere? And if so, how big are they? I’m thinking we could unzip and poke through them and see if there’s some weird circular dependency that’s taking up way too much disk space.

I checked, not sure if it is normal but inside the zipped balls as they call them, the node modules folder is really big and when looking into that, they all appear to be normal except for prisma that is 220mb

The bulk of it comes from the sdk and the engines folder:

Hmm I’ve got the SDK in there as a dependency of RR, but maybe it can be a devDependency: I’m pretty sure it’s only used on yarn rw record init. But then the datamodel.json file would have to get moved somewhere that can be checked in so that it’s available at deploy time and won’t need to be created by Netlify. Maybe it could go in api/src/models along with everything else? Or in api/src/db along with the other database stuff, but I don’t know if Prisma will freak out if there’s a file in there that it doesn’t manage itself.

So maybe just a change to:

and

And that’ll bring the bundle size back under control?

I can try that out see if it reduces the zipballs size locally. I like it going in api/src/models. I’ll keep you posted and if it works I’ll create a PR.

1 Like

Hi again. I’ve made some progress with this but I am a bit stuck at the moment.

So far I’ve:

  • moved the sdk to be a dev dependency
  • moved the location of the datamodel.json to the models folder as we said
  • Fixed the parseDatamodel method to ignore the file as it does with index.js for getting the model names
  • fixed prebuildApiFiles and findApiFiles to copy this json file to the pre-build

The next step would be to make sure that the json file is being correctly placed in the dist folder when running the build or dev commands.

Right now the issue I am seeing is that something is taking that .json file and transforming it into a js file with a source map associated file.

I am not very familiar wirth the build files so it is a bit hard for me to debug.

I can post what I have so far as a draft PR and we can collaborate there. WDYT?

1 Like

I’m betting it’s a Webpack and/or Babel config somewhere that’s doing this…and I’m no expert in those.

That would be great if you could open a PR and then I can pull in a couple of folks that are more familiar with Webpack/Babel to help!

Thanks for the work on this!

PR created :smile:

Update on this, it was fixed and should go out in the last v1-rc before code freeze

3 Likes

do you have a link to PR I have ran into a similar issue

Which version of RW are you running, this fix should have gone out with the V1. Can you provide more info on your error so I can help you debug?

Redwood 2.0.0 and here is some of my dependencies

"dependencies": {

    "@redwoodjs/forms": "2.0.0",

    "@redwoodjs/router": "2.0.0",

    "@redwoodjs/web": "2.0.0",

    "filepond": "^4.30.4",

    "filepond-plugin-image-exif-orientation": "^1.0.11",

    "filepond-plugin-image-preview": "^4.6.11",

    "nodemailer": "^6.7.7",

    "nodemailer-plugin-inline-base64": "^2.1.1",

    "prop-types": "15.8.1",

    "react": "17.0.2",

    "react-dom": "17.0.2",

    "react-filepond": "^7.1.2"

  },

Basically I am following the Netlify tutorial on the Docs on Redwood and I follow steps

Like using railway for postgres
then yarn rw setup deploy netlify

after making edits in netlify.toml file

[build]
command = "yarn rw deploy netlify"
publish = "web/dist"
# [functions]
functions = "api/dist/functions"
# directory = "api/dist/functions"
# http://localhost:8910/.netlify/functions/graphql
  # Flags "package-1" as an external node module for all functions.
  # [functions.functions]
  # external_node_modules = [
  #   "api/dist/",
  #   "lib",
  #   "directives",
  #   "services",
  #   "graphql"
  #   ]
  # Includes all Markdown files inside the "files/" directory.
  # included_files = ["files/*.md"]
# included_files = ["api/dist/*.js"]
# included_files = ["api/dist/directives/*"]
# included_files = ["api/dist/graphql/*"]
# included_files = ["api/dist/lib/*"]
# included_files = ["api/dist/services/*"]
# [functions]
# directory = "api/dist/"
  # [functions.directory2]
  #   directory2 = "api/dist/lib"
[dev]
  # yarn rw build --verbose && yarn rw prisma migrate deploy && yarn rw data-migrate up
  # To use [Netlify Dev](https://www.netlify.com/products/dev/),
  # install netlify-cli from https://docs.netlify.com/cli/get-started/#installation
  # and then use netlify link https://docs.netlify.com/cli/get-started/#link-and-unlink-sites
  # to connect your local project to a site already on Netlify
  # then run netlify dev and our app will be accessible on the port specified below
  framework = "redwoodjs"
  # Set targetPort to the [web] side port as defined in redwood.toml
  targetPort = 8910
  # Point your browser to this port to access your RedwoodJS app
  port = 8888
[[redirects]]
  from = "/*"
  to = "/200.html"
  status = 200

However my netlify serverless functions gives we error accessible at this URL

What I figured out is the serverless function cannot located the ‘…\lib\auth’ library even the build file in dist does have that library and code available. What this means is that all my graphQL queries are failing all forms all to submit data etc

http://diamondhaulingpa.com/.netlify/functions/graphql

I have been looking for answer all day sunday came cross this old post it was the most similar. The logs on netlify serverless function are pretty much the same

Aug 9, 08:34:31 PM: 342bc5ac Duration: 2322.17 ms	Memory Usage: 91 MB	Aug 9, 08:34:31 PM: Unknown application error occurred
Runtime.ImportModuleError
Aug 9, 08:34:32 PM: 2022-08-10T00:34:32.218Z	undefined	ERROR	Uncaught Exception 	{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module '..\\..\\lib\\auth'\nRequire stack:\n- /var/task/api/dist/directives/requireAuth/requireAuth.js\n- /var/task/api/dist/functions/graphql.js\n- /var/task/graphql.js\n- /var/runtime/index.mjs","stack":["Runtime.ImportModuleError: Error: Cannot find module '..\\..\\lib\\auth'","Require stack:","- /var/task/api/dist/directives/requireAuth/requireAuth.js","- /var/task/api/dist/functions/graphql.js","- /var/task/graphql.js","- /var/runtime/index.mjs","    at _loadUserApp (file:///var/runtime/index.mjs:726:17)","    at async Object.module.exports.load (file:///var/runtime/index.mjs:741:21)","    at async file:///var/runtime/index.mjs:781:15","    at async file:///var/runtime/index.mjs:4:1"]}
Aug 9, 08:34:32 PM: 5245799d Duration: 2251.47 ms	Memory Usage: 92 MB	Aug 9, 08:34:32 PM: Unknown application error occurred
Runtime.ImportModuleError
Aug 9, 08:34:33 PM: 2022-08-10T00:34:33.375Z	undefined	ERROR	Uncaught Exception 	{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module '..\\..\\lib\\auth'\nRequire stack:\n- /var/task/api/dist/directives/requireAuth/requireAuth.js\n- /var/task/api/dist/functions/graphql.js\n- /var/task/graphql.js\n- /var/runtime/index.mjs","stack":["Runtime.ImportModuleError: Error: Cannot find module '..\\..\\lib\\auth'","Require stack:","- /var/task/api/dist/directives/requireAuth/requireAuth.js","- /var/task/api/dist/functions/graphql.js","- /var/task/graphql.js","- /var/runtime/index.mjs","    at _loadUserApp (file:///var/runtime/index.mjs:726:17)","    at async Object.module.exports.load (file:///var/runtime/index.mjs:741:21)","    at async file:///var/runtime/index.mjs:781:15","    at async file:///var/runtime/index.mjs:4:1"]}
Aug 9, 08:34:33 PM: f7ff3246 Duration: 2368.25 ms	Memory Usage: 97 MB	Aug 9, 08:34:33 PM: Unknown application error occurred
Runtime.ImportModuleError

What are the “files/*.md”? Does the api import from a files directory are the same level as the api ? Or is files inside api?

You may need to force those to get built on Netlify by injecting them into the zip it creates.

See Error deploying to Netlify: Cannot find module '../../../.redwood/datamodel.json'

And

Yes so the included files is supposed to import those files outside of directory and I tried that directly for each folder and files contained in each folder but this did not work or remove the error.

What do you mean by force do you have a specific command line you are referencing?

# included_files = ["api/dist/*.js"]
# included_files = ["api/dist/directives/*"]
# included_files = ["api/dist/graphql/*"]
# included_files = ["api/dist/lib/*"]
# included_files = ["api/dist/services/*"]