Relative imports, babelRequireHook & frustrations

I wanted to post a thread about an issue I am having, and I figured this would be a good place to share this as it does require some context to understand.

I am currently setting up background tasks with the new redwood CLI exec feature. Under the hood the exec feature uses babelRequireHook to allow for importing arbitrary scripts by name and allow for those scripts to also import redwood related files such as db.js, services etc.

See the babelRequireHook in action here for the CLI exec command - redwood/exec.js at main · redwoodjs/redwood (github.com)

For my background tasks, I wanted to be able to run tasks in synchronous mode or asynchronous mode. This will help in development when you don’t want to always run the background worker.

This is my faktoryWorker.js script that I run with yarn rw exec faktoryWorker

/*global console,process,require*/

const faktory = require('faktory-worker')

import { adminReportEmailScheduledTask } from '../api/src/tasks/repeated'

faktory.register('repeated.adminReportEmailScheduledTask', async (taskArgs) => {

  await adminReportEmailScheduledTask(taskArgs)

})

export default async ({ _args }) => {

  const worker = await faktory

    .work({

      url: process.env.FAKTORY_URL,

    })

    .catch((error) => {

      console.error(`worker failed to start: ${error}`)

      process.exit(1)

    })

  worker.on('fail', ({ job, error }) => {

    console.error(`worker failed to start: ${error}`)

  })

}

When I run this, I run into a stack trace like so

[18:16:09] Generating Prisma client [started]
[18:16:12] Generating Prisma client [completed]
[18:16:12] Running script [started]
Error in script: Cannot find module '../../../../src/lib/db'
Require stack:
- /home/tharshan/Projects/uservitals/api/src/services/teams/team.js
- /home/tharshan/Projects/uservitals/api/src/tasks/repeated.js
- /home/tharshan/Projects/uservitals/scripts/faktoryWorker.js
- /home/tharshan/Projects/uservitals/node_modules/@redwoodjs/cli/dist/commands/exec.js
- /home/tharshan/Projects/uservitals/node_modules/@redwoodjs/cli/node_modules/yargs/index.cjs
- /home/tharshan/Projects/uservitals/node_modules/@redwoodjs/cli/dist/index.js
[18:16:15] Running script [completed]
Done in 9.90s.

If we follow the imports to trace the issue , let’s look at repeated.js

import { logger } from '../lib/logger'

import { db } from '../lib/db'

import { adminReportEmail, customerReportEmail } from '../services/teams/team'

Note I had to change these imports from ‘src/lib/logger’ to ‘…/lib/logger’ for the script to run, otherwise it will cause a similar exception and say Error in script: Cannot find module '../../../src/lib/logger'

in services/teams/team.js the imports are like so

import { db } from 'src/lib/db'

import { requireAuth, getOrCreateContact, deleteUser } from 'src/lib/auth'

Ive always used relative imports from ‘src’ and I think redwood already has an alias for it. but as you can see from the stack trace about, it tries to import it from '../../../../src/lib/db'

When this is run from the exec command - all the imports go nuts and it’s a game of whack a mole to track and fix it. If you change one import, another part breaks.

Can anyone suggest a solution here or shed light on why this is happening? Do we need to adjust the exec CLI feature and the babelRequireHook to handle imports differently?

1 Like

So actually writing out this post helped me think outside the box and dig deeper into how the babelRequireHook works for the redwood code base. In the end, I realised the imports inside the exec script was causing the issues, and fixed it by adding dynamic imports inside the script.

const { getPaths } = require('@redwoodjs/internal')

const faktory = require('faktory-worker')

import path from 'path'

import babelRequireHook from '@babel/register'

babelRequireHook({

  extends: path.join(getPaths().api.base, '.babelrc.js'),

  extensions: ['.js', '.ts'],

  plugins: [

    [

      'babel-plugin-module-resolver',

      {

        alias: {

          src: getPaths().api.src,

        },

      },

    ],

  ],

  ignore: ['node_modules'],

  cache: false,

})

const runTask = async (taskName, taskArgs) => {

  const [module, fn] = taskName.split('.')

  const taskPath = path.join(getPaths().api.src, 'tasks', `${module}.js`)

  const script = await import(taskPath)

  await script[fn](taskArgs)

}

The above lines of code allow runTask to import any files within the api/tasks folder which is where I keep all my async stuff.

2 Likes

So ive run into another issue with my shared packages using yarn workspace.

I have a packages folder at the root level of the project.

image

I include these using my package.json

image

This allows the web and api side to share the same files. I do create ejs and esm copies of the files.

Essentially the difference is export { splitIdentifier, vs module.exports = { splitIdentifier, in the files.

This works for both web and api.

However, I recently found that my faktoryWorker using babelRequireHook has issues with this.

Here is my faktoryWorker.js

/*global console,process,require*/

const { getPaths } = require('@redwoodjs/internal')

const faktory = require('faktory-worker')

import path from 'path'

import babelRequireHook from '@babel/register'

console.log(getPaths().base)

babelRequireHook({

  extends: path.join(getPaths().api.base, '.babelrc.js'),

  extensions: ['.js', '.ts'],

  plugins: [

    [

      'babel-plugin-module-resolver',

      {

        alias: {

          src: getPaths().api.src,

          '@uservitals/url': path.join(getPaths().base, 'packages', 'url', 'url-cjs.js'),

          '@uservitals/constants': path.join(getPaths().base, 'packages', 'constants', 'constants-cjs.js'),

          '@uservitals/utils': path.join(getPaths().base, 'packages', 'utils', 'utils-cjs.js'),

        },

      },

    ],

  ],

  ignore: ['node_modules'],

  cache: false,

})

const runTask = async (taskName, taskArgs) => {

  const [module, fn] = taskName.split('.')

  const taskPath = path.join(getPaths().api.src, 'tasks', `${module}.js`)

  const script = await import(taskPath)

  await script[fn](taskArgs)

}

faktory.register('auth.postSignupTask', async (taskArgs) => {

  await runTask('auth.postSignupTask', taskArgs)

})

faktory.register('auth.postUpdateStoryStatus', async (taskArgs) => {

  await runTask('auth.postUpdateStoryStatus', taskArgs)

})

faktory.register('repeated.adminReportEmailScheduledTask', async (taskArgs) => {

  await runTask('repeated.adminReportEmailScheduledTask', taskArgs)

})

faktory.register('repeated.customerVoiceEmailScheduledTask', async (taskArgs) => {

  await runTask('repeated.customerVoiceEmailScheduledTask', taskArgs)

})

faktory.register('repeated.changelogNotificationEmailScheduledTask', async (taskArgs) => {

  await runTask('repeated.changelogNotificationEmailScheduledTask', taskArgs)

})

export default async ({ _args }) => {

  const worker = await faktory

    .work({

      url: process.env.FAKTORY_URL,

    })

    .catch((error) => {

      console.error(`worker failed to start: ${error}`)

      process.exit(1)

    })

  worker.on('fail', ({ _job, error }) => {

    console.error(`worker failed to start: ${error}`)

  })

}

Unfortunately now when asyncTask is used and run by the worker, any imported files that use @packages/url or others causes an error

faktory-worker:connection SEND: FAIL {"message":"Cannot use import statement outside a module","backtrace":["/home/tharshan/Projects/uservitals/packages/url/url-cjs.js:1","import _forEachInstanceProperty from \"@babel/runtime-corejs3/core-js/instance/for-each\";","^^^^^^","","SyntaxError: Cannot use import statement outside a module"," at wrapSafe (internal/modules/cjs/loader.js:1001:16)"," at Module._compile (internal/modules/cjs/loader.js:1049:27)"," at Module._compile (/home/tharshan/Projects/uservitals/node_modules/pirates/lib/index.js:99:24)"," at Module._extensions..js (internal/modules/cjs/loader.js:1114:10)"," at Object.newLoader [as .js] (/home/tharshan/Projects/uservitals/node_modules/pirates/lib/index.js:104:7)"," at Module.load (internal/modules/cjs/loader.js:950:32)"," at Function.Module._load (internal/modules/cjs/loader.js:790:14)"," at Module.require (internal/modules/cjs/loader.js:974:19)"," at require (internal/modules/cjs/helpers.js:92:18)"," at Object.<anonymous> (/home/tharshan/Projects/uservitals/api/src/slack/notifications.js:3:1)"],"jid":"1f968782-51c8-4ac1-ab24-2121aad114ab"} +0ms