Deploy failed due to an error in @netlify/plugin-functions-core plugin

Get Help and Help Others
1

I am new to TypeScript, Netlify, and RedwoodJS.

Yesterday I ran into a lot of problems trying to deploy what I thought was a simple RedwoodJS app without a database. (See How to fix graphql 404 on Netlify)

Possible to deploy without a database? looked like it would be helpful but actually:

  • I did not need to delete the api/prisma directory
  • I did not comment out the contents of api/src/lib/db.js
  • In netlify.toml, I commented out the [[plugins]] heading and everything underneath it. I also used command = "yarn rw build" under [build]
  • At Netlify App, I changed Build command
    to yarn workspaces run build. I don’t understand how this relates to the netlify.toml setting or which takes precedence.

Then the other deployment error I was having was:

Deploy failed due to an error in @netlify/plugin-functions-core plugin.
Plugin “@netlify/plugin-functions-core” internal error

Error: In file “/opt/build/repo/api/dist/functions/graphql.js”: Cannot find module ‘./platformCombiner.ts’ from ‘/opt/build/repo/api/dist/services’

In “onBuild” event in “@netlify/plugin-functions-core” from core at /opt/buildhome/.netlify-build-nvm/versions/node/v12.16.3/lib/node_modules/@netlify/build/node_modules/resolve/lib/async.js:142:35 at load (/opt/buildhome/.netlify-build-nvm/versions/node/v12.16.3/lib/node_modules/@netlify/build/node_modules/resolve/lib/async.js:161:43) at onex (/opt/buildhome/.netlify-build-nvm/versions/node/v12.16.3/lib/node_modules/@netlify/build/node_modules/resolve/lib/async.js:186:17) at /opt/buildhome/.netlify-build-nvm/versions/node/v12.16.3/lib/node_modules/@netlify/build/node_modules/resolve/lib/async.js:13:69 at FSReqCallback.oncomplete (fs.js:167:21)

Error properties: { code: ‘MODULE_NOT_FOUND’ }

9:56:37 PM: ┌────────────────────────────────────────────────────────┐
9:56:37 PM: │ Plugin "@netlify/plugin-functions-core" internal error │
9:56:37 PM: └────────────────────────────────────────────────────────┘
9:56:37 PM: ​
9:56:37 PM:   Error message
9:56:37 PM:   Error: In file "/opt/build/repo/api/dist/functions/graphql.js": Cannot find module './platformCombiner.ts' from '/opt/build/repo/api/dist/services'

Here is what I figured out. Here are 3 attempts that each worked locally but then caused errors when trying to deploy to Netlify:

  • api/src/services/orders.js had import { combinePlatforms, decimalPlaces } from './platformCombiner.ts'; because I prefer splitting my code into small files
  • api/src/services/orders.ts had import { combinePlatforms, decimalPlaces } from './platformCombiner.ts'; because I wondered if both files needed to be TypeScript, so I converted orders.js to orders.ts
  • I also tried moving platformCombiner.ts out to a separate directory called “helpers” (a sibling of the “services” directory) so that platformCombiner.ts wouldn’t be considered its own service.

I figured out a workaround, but I don’t understand why this was necessary:

The workaround was merge platformCombiner.ts into orders.ts so that there was just one larger file.

Thanks for any ideas. :smiley:

UPDATE: To clarify, the main part of my question is: Why was I not able to have my orders.ts service import platformCombiner.ts either in the same directory or in a sibling directory? Why did I need to merge platformCombiner.ts into orders.ts?

2

Just to clarify … deploying w/out a database is different than deploying without an api side.

I could see many situations where I would still want:

  • graphql
  • lambda functions
  • services / sdl
  • auth

but not have a “database” or use Prisma.

I could use services to talk to other apis or GraphQL servers, etc.

I could have a web app that just uses auth to protect certain pages, etc.

I’ve never tried this, to be honest, but I think I would (first) simply change my build command on Netlify from

yarn rw build && yarn rw db up --no-db-client --auto-approve && yarn rw dataMigrate up

to just

yarn rw build

don’t understand how this relates to the netlify.toml setting or which takes precedence.

The command specified in the netlify.toml will override the one set in the Netlify build settings UI. This will let you specify different build commands per deploy (like production, deploy-preview, branch etc).

image
image899×354 24.4 KB

3

Thanks, @dthyresson. Yep, my app is an example of a case where I wanted the API side but no database. I was using fetch to pull from third-party APIs and then make it available via GraphQL to my front-end, like the weather example at https://redwoodjs.com/cookbook/using-a-third-party-api

4

@ryancwalsh

I wanted to give this a quick test … and I was successful, but I do see an issue.

Can see the app live at https://redwoodjs-no-db.netlify.app/ and the repo is public at:

image

This example still uses the api and GraphQL to return “messages” and a link to invoke a function.

  • message for an id/text
  • list of messages
  • call a hello function

But, I see that when building the api side it tries to generate the DB client.

Locally:

  ✖ Generating the Prisma client...
    → info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Command failed with exit code 1: yarn prisma generate
Error: 
You don't have any models defined in your schema.prisma, so nothing will be generated.

And on Netlify build

1:09:48 PM: $ /opt/build/repo/node_modules/.bin/rw build
1:09:49 PM: Generating the Prisma client... [started]
1:09:51 PM: Generating the Prisma client... [failed]
1:09:51 PM: → Command failed with exit code 1: yarn prisma generate
1:09:51 PM: Error: 
1:09:51 PM: You don't have any models defined in your schema.prisma, so nothing will be generated.
You can define a model like this:
1:09:51 PM: model User {
1:09:51 PM:   id    Int     @id @default(autoincrement())
1:09:51 PM:   email String  @unique
1:09:51 PM:   name  String?
1:09:51 PM: }
1:09:51 PM: More information in our documentation:
1:09:51 PM: https://pris.ly/d/prisma-schema
1:09:51 PM: error Command failed with exit code 1.
1:09:51 PM: $ /opt/build/repo/node_modules/.bin/prisma generate
1:09:51 PM: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
1:09:51 PM: Command failed with exit code 1: yarn prisma generate
1:09:51 PM: Error: 
1:09:51 PM: You don't have any models defined in your schema.prisma, so nothing will be generated.

But, my site was still deployed and works as expected.

I’ll see how to build the api side w/out the Prisma client.

1 Like
5

Oh, I had just commented out my Prisma schema … looks like:

export const handler = async ({ verbose = true, force = true }) => {
  if (!fs.existsSync(getPaths().api.dbSchema)) {
    console.log(
      `Skipping database and Prisma client generation, no \`schema.prisma\` file found: \`${
        getPaths().api.dbSchema
      }\``
    )
    return
  }

It will skip this if none present; let’s check … … :mantelpiece_clock: … … :mantelpiece_clock: …

Nice!

Skipping database and Prisma client generation, no `schema.prisma` file found: `/x/redwoodjs/discord-help/no_db/api/prisma/schema.prisma`
  ✔ Building "api"...
  ✔ Building "web"...
✨  Done in 14.37s.

And on Netlify … :mantelpiece_clock: …

Nice!

1:22:51 PM: ┌────────────────────────────────────┐
1:22:51 PM: │ 1. build.command from netlify.toml │
1:22:51 PM: └────────────────────────────────────┘
1:22:51 PM: ​
1:22:51 PM: $ yarn rw build
1:22:51 PM: yarn run v1.22.4
1:22:51 PM: $ /opt/build/repo/node_modules/.bin/rw build
1:22:52 PM: Skipping database and Prisma client generation, no `schema.prisma` file found: `/opt/build/repo/api/prisma/schema.prisma`
1:22:52 PM: [17:22:52] Building "api"... [started]
1:22:55 PM: [17:22:55] Building "api"... [completed]
1:22:55 PM: [17:22:55] Building "web"... [started]
1:23:11 PM: [17:23:11] Building "web"... [completed]
1:23:11 PM: Done in 19.98s.
1:23:11 PM: ​
1:23:11 PM: (build.command completed in 20.2s)
1:23:11 PM: ​
1:23:11 PM: ┌────────────────────────────────────────────────────────┐
1:23:11 PM: │ 2. onBuild command from @netlify/plugin-functions-core │
1:23:11 PM: └────────────────────────────────────────────────────────┘
1:23:11 PM: ​
1:23:11 PM: Packaging Functions from api/dist/functions directory:
1:23:11 PM:  - graphql.js
1:23:11 PM:  - hello.js
1:23:35 PM: ​
1:23:35 PM: (@netlify/plugin-functions-core onBuild completed in 24.7s)
1:23:36 PM: ​
1:23:36 PM: ┌─────────────────────────────┐
1:23:36 PM: │   Netlify Build Complete    │
1:23:36 PM: └─────────────────────────────┘
1:23:36 PM: ​
1:23:36 PM: (Netlify Build completed in 45.8s)

So, that was the key for me – removing the prisma directory.

2 Likes
6

@dthyresson I really appreciate all the effort you’ve been putting into this!

To clarify, the main part of my question is: Why was I not able to have my orders.ts service import platformCombiner.ts either in the same directory or in a sibling directory? Why did I need to merge platformCombiner.ts into orders.ts (all as one bigger file)?

It’s quite possible that I’m misunderstanding something very basic.

1 Like
7

Ah – well that’s what happens when I don’t read the item properly (or I do at like 4am) :slight_smile:

But, I did want to try out that scenario and turn it into a Cookbook, so this was a good way of making be try it out.

Two things:

  1. Can you share the repo if public – or at least the code for the sdl and services and helpers?

  2. You can test a local netlify build locally

Just

npm install netlify-cli -g

setup authorization etc and then

  • netlify link to connect to your site

  • netlify build

Note: not all netlify cli command work with RedwoodJS … for example, the netlify dev does not (yet).

It will then build just like it will when deploying … including the plugins, etc.

  ✔ Building "api"...
  ✔ Building "web"...
✨  Done in 15.44s.

(build.command completed in 15.6s)

┌────────────────────────────────────────────────────────────┐
│ 2. onPostBuild command from @netlify/plugin-functions-core │
└────────────────────────────────────────────────────────────┘

Packaging Functions from api/dist/functions directory:
 - graphql.js
 - hello.js

(@netlify/plugin-functions-core onPostBuild completed in 22.1s)

┌─────────────────────────────┐
│   Netlify Build Complete    │
└─────────────────────────────┘

Which is a nice way to test w/out deploying.

If you’re able to share some of that code, maybe we can sort things out.

Cheers!

2 Likes
8

@dthyresson Cool, I didn’t know about https://docs.netlify.com/cli/get-started/#installation

Here is my code: https://github.com/ryancwalsh/crypto_platform/tree/main/api/src/services

Thanks!

9

I cloned your repo and made did a test and of course I get the

  "message": "Cannot read property 'buy' of null",

So, I need to know what to serch for … so I looked at Bittrex and noticed you are using v1.1. of their api:

Warning: The v1.1 API will reach its planned end of life and no longer be supported as of 9/30/2020.

Please upgrade to the v3 API prior to that date.

which will be EOL… well, a few weeks ago?

But looks like BTC-LTC is a valid marketPair.

https://api.bittrex.com/api/v1.1/public/getorderbook?market=BTC-LTC&type=both

But you are converting underscores, so I did query:

image
image886×552 50 KB

I was able to move things into helpers, but I am not a TS expeort but I do know that the use of function and the semicolons strikes me as things I have not seen. And the ESLint isn’t cleaning up my code.

Here’s what I have in api:

src/services/orders.ts

import { fetchBittrex,
  fetchPoloniex,
  getCleanedPoloniexResponse,
  getCleanedBittrexResponse,
  combinePlatforms } from 'src/lib/orderUtils'


export const getMarket = async ({ marketPair }): Promise<Market> => {
  console.log('getMarket');
  const bittrexResponse = await fetchBittrex(marketPair);
  const bittrexCleaned = getCleanedBittrexResponse(bittrexResponse);
  const poloniexResponse = await fetchPoloniex(marketPair);
  const poloniexCleaned = getCleanedPoloniexResponse(poloniexResponse);
  const platformsCombined = combinePlatforms([bittrexCleaned, poloniexCleaned]);
  return platformsCombined;
};

src/lib/orderUtils.ts

Everything else …

import fetch from 'node-fetch';
import exactMath from 'exact-math';

const decimalPlaces = 10;

function combineType(name: string, orders: Array<Order>, ratesObj: Rates): Rates {
  const result = { ...ratesObj };
  orders.forEach((order) => {
    const rate = Number(order.rate).toFixed(decimalPlaces);
    const platform = {
      name,
      quantity: Number(order.quantity).toFixed(decimalPlaces),
    };
    if (Object.keys(result).includes(rate)) {
      result[rate].total = Number(exactMath.add(result[rate].total, order.quantity)).toFixed(decimalPlaces);
      result[rate].platforms.push(platform);
    } else {
      result[rate] = {
        rate,
        platforms: [platform],
        total: Number(order.quantity).toFixed(decimalPlaces),
      };
    }
  });
  return result;
}

function convertToArray(rates: Rates, platformNames: Array<string>): Array<Order> {
  // Convert rates objects to arrays, and make sure that there is a quantity for every platform for every rate:
  const orders = [];
  Object.keys(rates).forEach((key) => {
    const rateEntry = rates[key];
    const platformNamesToCheck = [...platformNames];
    rateEntry.platforms.forEach((platform: PlatformQuantity) => {
      platformNamesToCheck.splice(platformNamesToCheck.indexOf(platform.name), 1); // remove from array
    });
    platformNamesToCheck.forEach((name) => {
      rateEntry.platforms.push({
        name,
        quantity: Number(0).toFixed(decimalPlaces),
      });
    });
    rateEntry.platforms.sort((a, b) => {
      // This sort order must match that of the `platforms` array in client-side web/src/components/MarketCell/MarketCell.js
      if (a.name > b.name) {
        return 1;
      } else if (b.name > a.name) {
        return -1;
      } else {
        return 0;
      }
    });
    orders.push(rateEntry);
  });
  return orders;
}

export function combinePlatforms(platformResponses: Array<Platform>): Market {
  let bidRates = {};
  let askRates = {};
  platformResponses.forEach((platform: Platform) => {
    bidRates = combineType(platform.name, platform.bids, bidRates);
    askRates = combineType(platform.name, platform.asks, askRates);
  });
  // console.log('bidRates', JSON.stringify(bidRates, null, 2), 'bidRates');
  const platformNames = platformResponses.map((platform: Platform) => {
    return platform.name;
  });
  const bids = convertToArray(bidRates, platformNames);
  const asks = convertToArray(askRates, platformNames);
  bids.sort((a, b) => {
    return Number(b.rate) - Number(a.rate);
  });
  asks.sort((a, b) => {
    return Number(a.rate) - Number(b.rate);
  });
  const combinedResult = {
    bids,
    asks,
  };
  // console.log('combinedResult', JSON.stringify(combinedResult));
  return combinedResult;
}

export function getCleanedBittrexResponse(bittrexResponse: BittrexResponse): Platform {
  const result: Platform = {
    name: 'bittrex',
    bids: bittrexResponse.result.buy.map((order) => {
      return {
        rate: exactMath.round(order.Rate, -decimalPlaces),
        quantity: exactMath.round(order.Quantity, -decimalPlaces),
      };
    }),
    asks: bittrexResponse.result.sell.map((order) => {
      return {
        rate: exactMath.round(order.Rate, -decimalPlaces),
        quantity: exactMath.round(order.Quantity, -decimalPlaces),
      };
    }),
  };
  // console.log('getCleanedBittrexResponse result', JSON.stringify(result, null, 2));
  return result;
}

export function getCleanedPoloniexResponse(poloniexResponse: Platform): Platform {
  const [rateIndex, quantityIndex] = [0, 1];
  const result = { ...poloniexResponse };
  result.bids = result.bids.map((order) => {
    return {
      rate: exactMath.round(order[rateIndex], -decimalPlaces),
      quantity: exactMath.round(order[quantityIndex], -decimalPlaces),
    };
  });
  result.asks = result.asks.map((order) => {
    return {
      rate: exactMath.round(order[rateIndex], -decimalPlaces),
      quantity: exactMath.round(order[quantityIndex], -decimalPlaces),
    };
  });
  result.name = 'poloniex';
  // console.log('getCleanedPoloniexResponse result', JSON.stringify(result, null, 2));
  return result;
}

export async function fetchBittrex(marketPair: string): Promise<BittrexResponse> {
  console.log('fetchBittrex', marketPair);
  const bittrexMarketPair = marketPair.replace('_', '-');
  const bittrexApiResponse = await fetch(`https://api.bittrex.com/api/v1.1/public/getorderbook?market=${bittrexMarketPair}&type=both`); // See https://bittrex.github.io/api/v1-1
  const bittrexResult = await bittrexApiResponse.json();
  // console.log('bittrexResult', JSON.stringify(bittrexResult));
  return bittrexResult;
}

export async function fetchPoloniex(marketPair: string): Promise<Platform> {
  console.log('fetchPoloniex', marketPair);
  const poloniexApiResponse = await fetch(`https://poloniex.com/public?command=returnOrderBook&currencyPair=${marketPair}&depth=100`); // See https://docs.poloniex.com/#returnorderbook
  const poloniexResult = await poloniexApiResponse.json();
  // console.log('poloniexResult', JSON.stringify(poloniexResult, null, 2), 'poloniexResult');
  return poloniexResult;
}

Web app is workign ok, too.

I did not try a Netlify build as I did not have a site to link.

You may be able to not export as many functions from the orderUtil.ts and do more parsing and assembly there and have a smaller service, but I did not look into the parsing and such.

BTW I just renamed orderUtil to platformCombiner and still ok.

I’d be curious if you can deploy to Netlify with that.

2 Likes
10

Thank you so much for looking into that. :smiley: I’m impressed.

To clarify my original question above, I just changed “Here are 3 attempts that each did NOT work:” to “Here are 3 attempts that each worked locally but then caused errors when trying to deploy to Netlify:”.

So if you didn’t try deploying to Netlify, you probably wouldn’t have run into the problem I was seeing.

And maybe this “question” belongs in a Netlfy forum instead, but I posted it here since my feeling has been that they are very related (and I’ve still only ever heard of Netlify in the context of RedwoodJS).

11

So I checked in that repo (https://github.com/dthyresson/bittrex-test) and built with Netlify locally via netlify build:

  ✔ Generating the Prisma client...
  ✔ Building "api"...
  ✔ Building "web"...
✨  Done in 19.46s.

(build.command completed in 19.6s)

┌────────────────────────────────────────────────────────────┐
│ 2. onPostBuild command from @netlify/plugin-functions-core │
└────────────────────────────────────────────────────────────┘

Packaging Functions from api/dist/functions directory:
 - graphql.js

(@netlify/plugin-functions-core onPostBuild completed in 15.6s)

and the deploy on Netlify was ok, too:

3:53:29 PM: ┌────────────────────────────────────┐
3:53:29 PM: │ 1. build.command from netlify.toml │
3:53:29 PM: └────────────────────────────────────┘
3:53:29 PM: ​
3:53:29 PM: $ yarn rw build
3:53:30 PM: yarn run v1.22.4
3:53:30 PM: $ /opt/build/repo/node_modules/.bin/rw build
3:53:31 PM: Generating the Prisma client... [started]
3:53:33 PM: Generating the Prisma client... [completed]
3:53:33 PM: [19:53:33] Building "api"... [started]
3:53:36 PM: [19:53:36] Building "api"... [completed]
3:53:36 PM: [19:53:36] Building "web"... [started]
3:54:01 PM: [19:54:01] Building "web"... [completed]
3:54:01 PM: Done in 31.48s.
3:54:01 PM: ​
3:54:01 PM: (build.command completed in 31.7s)
3:54:01 PM: ​
3:54:01 PM: ┌────────────────────────────────────────────────────────┐
3:54:01 PM: │ 2. onBuild command from @netlify/plugin-functions-core │
3:54:01 PM: └────────────────────────────────────────────────────────┘
3:54:01 PM: ​
3:54:01 PM: Packaging Functions from api/dist/functions directory:
3:54:01 PM:  - graphql.js
3:54:38 PM: ​
3:54:38 PM: (@netlify/plugin-functions-core onBuild completed in 37.1s)
3:54:38 PM: ​
3:54:38 PM: ┌─────────────────────────────┐
3:54:38 PM: │   Netlify Build Complete    │
3:54:38 PM: └─────────────────────────────┘
3:54:38 PM: ​
3:54:38 PM: (Netlify Build completed in 1m 9.9s)
3:54:38 PM: Caching artifacts
3:54:38 PM: Started saving node modules
3:54:39 PM: Finished saving node modules
3:54:39 PM: Started saving build plugins
3:54:39 PM: Finished saving build plugins
3:54:39 PM: Started saving yarn cache
3:54:39 PM: Finished saving yarn cache
3:54:39 PM: Started saving pip cache
3:54:39 PM: Finished saving pip cache
3:54:39 PM: Started saving emacs cask dependencies
3:54:39 PM: Finished saving emacs cask dependencies
3:54:39 PM: Started saving maven dependencies
3:54:39 PM: Finished saving maven dependencies
3:54:39 PM: Started saving boot dependencies
3:54:39 PM: Finished saving boot dependencies
3:54:39 PM: Started saving go dependencies
3:54:39 PM: Finished saving go dependencies
3:54:42 PM: Build script success
3:54:42 PM: Starting to deploy site from 'web/dist'
3:54:43 PM: Creating deploy tree 
3:54:43 PM: Creating deploy upload records
3:54:43 PM: 0 new files to upload
3:54:43 PM: 1 new functions to upload
3:54:49 PM: Starting post processing
3:54:49 PM: Post processing - HTML
3:54:49 PM: Post processing - header rules
3:54:49 PM: Post processing - redirect rules
3:54:49 PM: Post processing done
3:54:49 PM: Site is live

Note: I’ll delete the repo/site in a day or so.

If you are in the Discord chat, DM me and I’ll share the site url. I am getting data from the Bittrex API in the tables.

1 Like
12

In case it helps anyone reading this older thread, see also Typescript prod deployment

Removing the “.ts” from the import lines fixed the Netlify production build. :muscle:

1 Like