Separate generated services and SDL files from custom ones

[Sebastian]

I have a question that docs do no answer now:
how can I create custom service and SDL / GraphQL query ex.

I have user(id: String!) query and users (empty) query
I want to add usersBy(name: String!) query.

I can create an SDL in separate directory and it will be picked up by RW, merged and … complain that service is not implemented.
IF I implement the service in src/services/users.ts then all is good but these changes will be lost as soon as I need to regenerate CRUD operations.

If I put the service implementation in any other folder or file it will still complain. So how can I connect graphQL query/mutation etc. with service function?

Thank you!

[dthyresson]

@Sebastian

If I put the service implementation in any other folder or file it will still complain. So how can I connect graphQL query/mutation etc. with service function?

Hi @Sebastian you should absolutely be able to creat a new sdl and a new service for that userBy query operation.

I do that all the time … say for a userManager service.

That way, as you said, the generators won’t overwrite your standard CRUD generated files.

Perhaps reloading VSCode or restarting Typescript helps — or restarting the dev server?

RedwoodJS will load all files in the sdl directory and in the service (just keep the same naming conventions as the generated files) and merges them and maps the operation userBy to the userBy method.

[Sebastian]

Thank you @dthyresson, I checked again with GraphQL client and it seems I found a bug in the Redwood plugin.

Here’s my folder and file structure and what I tried:

Custom schema definition is in subfolder to keep it separated from generated ones it’s located in
[root]/api/src/graphql/custom/usersCustom.sdl.ts:

It defines usersBy field since users is already taken by generated one and overriding is not possible as far as I know.

export const schema = gql`
  type Query {
    usersBy(firstName: String): [User!]! @requireAuth
  }
`;

If there is no service Redwood (plugin) will report SERVICE_NOT_IMPLEMENTED error on that line.

The service is defined as separate file in:
[root]/api/services/custom/usersCustom.ts

import type { QueryResolvers } from 'types/graphql';

import { db } from 'src/lib/db';

export const usersBy: QueryResolvers['usersBy'] = ({ firstName }) => {
  return db.user.findMany({
    where: {
      firstName: {
        contains: firstName,
        mode: 'insensitive',
      },
    },
  });
};

As long as I export variable usersBy it will work actually, BUT the plugin still reports the error about service not being implemented. I trusted it too much and probably had errors while testing in the beginning.

Some more context for those starting journey with Redwood and services:

export const usersBy: QueryResolvers['usersBy'] = ({ firstName }) => { ... }

1. exported variable MUST be named just like the query in SDL
   @dthyresson Q1: is there a way to manually map the SDL to service?

2. QueryResolvers will be generated with types (yarn rw g types) and should reference name of the SDL query/field so that parameters of the query are known by TypeScript (for autocomplete etc.)

3. service file can be located in any depth in folder structure inside services folder.

The error is still shown in VS Code that service is not implemented.

@dthyresson Please correct me if I’m wrong about the above.

Q4: I also see that when generating SDL sometimes (cant pinpoint why yet) ModelResolvers get deleted completely. Maybe it’s when I generate service on it’s own separately from SDL (not sure) or there are “errors” in SDL(?). I just now regenerated user with SDL and it would add UserResolvers that were not there a sec ago at all. Why would that happen @dthyresson ?

Offtopic:

@dthyresson I also have a request (humble one ) I have posted my problems with integrating custom components (for MultiSelect) is there someone I could ask for help from the team to take a look? I know you have little time and I do appreciate help but that thing is driving me crazy since last few days and I’m stuck.
ex. Integrate custom select (react-select or other) as custom Redwood component

I did study the docs, used react-hook-form register function but had no luck, either API failed or components are not sending data at all.

I have also seen that Mantine is integrated and awaits release, when could we expect that and is there a code in RW that would show how such components should be integrated?

[dthyresson]

Apologies - I realized I was not clear in my last post.

All sdl must be in the generated sdl directory

All services must be in the generated services directory in api.

This is because of how the sdl and services are imported and passed when creating the GraphQLHandler.

redwoodjs/redwood/blob/259fd77534800fbf0751a08f4d5ff35effb399a1/packages/create-redwood-app/template/api/src/functions/graphql.ts#L4

See lines 4-5 and then how these glob imports are passed.

If you really want to have your sdl somewhere else (not recommended) then you have to manually import and merge in the rest when creating the handler.

Let me know if that works?

[Sebastian]

Thanks @dthyresson - yeah I can have subfolders (it works) since glob allows it with ‘src/graphql/**/*.sdl.{js,ts}’ (part with /**/)

but just now I changed from usersBy to usersByFirstName both in SDL definition and in service. Generated types again with yarn rw g types and… still usersBy is shown in generated graphql.d.ts

What am I missing I did manage to generate files properly just an hour ago or so verifying your last reply…

Types generated properly, no errors on dev server restart.

I changed the name as I was adding another custom query and that did not generate or work.

[dthyresson]

Type generation looks in those specific folders which iAd why I did not recommend moving these tk any custom directory — but wanted to explain how it might work.

I do not see any benefit from deviating from the existing Redwood app pattern for the handler.

[Sebastian]

I am using those folders… placing all files in \graphql\...\*.sdl.ts or in \services\...\*.ts files.

Or do I need to put them strictly in the root with no subfolders as the glob pattern looks inside subfolders as well it should not be a problem as long as they are in graphql and services folders right?

I do not want to deviate from RW way, far from it

BTW. this is so strange…

1. dev server is running
2. I created the second SDL and Service
3. Put all service files in /services/custom/ folder (2 files named: usersCustom.ts and exercisesCustom.ts)
4. Put all SDL files in /graphql/custom/ folder (2 files named usersCustom.sdl.ts and exercisesCustom.sdl.ts

(When I wrote the post above I renamed folder from custom to _custom and maybe that affected something? I changed back and now:
The "exercisesCustom.*.ts files work and I can rename both the SDL query as well as service - as long as they are named the same it works.

For userByFirstName query and service named the same it fails. Nothing I do will generate new name it’s stuck with usersBy and that sequence is nowhere in any files in the project (I even deleted files from .redwood and dist folders as they used old name)

[Sebastian]

@dthyresson - please let me know if this is correct or am I missing anything to understand the process
PS. can such explanation be added to docs so it’s clear what should be done in similar situations?

OK I think I have it figured out mostly:

Let’s assume that in /api/graphql/custom/usersCustom.sdl.ts I have this code (full file):

export const schema = gql`
  type Query {
    findUsersBy(firstName: String): [User]! @requireAuth
  }
`;

and in api/services/custom/usersCustom.ts:

import type { QueryResolvers } from 'types/graphql';

import { db } from 'src/lib/db';

export const findUsersBy: QueryResolvers['findUsersBy'] = ({ firstName }) => {
  return db.user.findMany({
    where: {
      firstName: {
        contains: firstName,
        mode: 'insensitive',
      },
    },
  });
};

1. When dev server is running and I make any manual changes to SDL or service files they are picked up and reloaded on the dev server (in memory) but not in the code this is why things work ex. I can rename the SDL ex. to (without changing QueryResolver to new name) ex. to

seekUsersBy(firstName: String): [User]! @requireAuth

without touching the service and GraphQL client will be able to reload with new query name but it will fail as the service is not present.
2. If I then rename the service without changing the QueryResolver like so:

export const seekUsersBy: QueryResolvers['findUsersBy'] = ({ firstName }) => {

it will now return correct data and IDE will not complain as changes were picked up by the server.

Now to change this properly I need to change the QueryResolver type to seekUsersBy:

export const seekUsersBy: QueryResolvers['seekUsersBy'] = ({ firstName }) => {

This will make IDE complain as the graphql.d.ts (in \api\types directory) has not been changed yet. All changes are in memory/for dev server only.

to fix this I must manually run yarn rw g types. All errors will now go away.

I am not sure what I did wrong previously but this process seems to work consistently for now.

[dthyresson]

@Danny Could you help answer @Sebastian 's questions about the QueryResolver types here as I know you’re the expert here after to work you’ve done recently.

Thanks.

@Sebastian

it will now return correct data and IDE will not complain as changes were picked up by the server.

Now to change this properly I need to change the QueryResolver type to seekUsersBy:

export const seekUsersBy: QueryResolvers['seekUsersBy'] = ({ firstName }) => {

This will make IDE complain as the graphql.d.ts (in \api\types directory) has not been changed yet. All changes are in memory/for dev server only.

to fix this I must manually run yarn rw g types. All errors will now go away.

I am not sure what I did wrong previously but this process seems to work consistently for now

[danny]

Hello @Sebastian - I think I understand the problem you’re seeing here, but just to be crystal clear about it, would it be possible to provide a reproduction repo or (preferably) a git pod snapshot? See steps here redwood/CONTRIBUTING.md at main · redwoodjs/redwood · GitHub

[Sebastian]

@danny
Hi,
last Friday I created a separate repo to show issues we had with Redwood (seems more like GraphQL API side that I do not understand how to make work).
I added an example of issue that I did describe above here:

GitHub - morecode-spzoo/redwood-playground: Test repository for RedwoodJS

There are 2 issues:

1. The one describe above and I was unable to reproduce it in this repo, in our main one we have more dependencies (tailwind ui, i18n).
2. The one NOT mentioned above but reproduceable: SERVICE_NOT_IMPLEMENTED error on SDL with custom file for service code.

Readme has all the explanations and all code is tagged (everything is in main repo… no branches needed for now).

We plan to maintain it to show issues and how to fix them. I do hope that we can actually resolve and understand the issues and maybe contribute to the docs at least when it’s all understood.

Thank you!

[dthyresson]

Thanks for the repo and I’ll have a look.

That said, you have a different pattern:

There’s no need for a “custom” subdirectory.

Just have the sdl files in the same directory as there others:

Same with the services:

I actually find the use of “custom” confusing.

I typically will create ModelManager or something like that sdl and service.

So, a BooksManager or a UserQueryManager and have all my non-CRUD auto generated graphql schema and resolvers there.

And organize the directories as per the generated content.

[dthyresson]

You can fix the service unimplemented warning by reorganizing your services to follow the naming convention:

This is the recommended directory structure since it is a nice way of organizing your service, test and scenarios in one easy to find place.

[dthyresson]

FYI, the reason VSCode complains is that

// redwood/packages/structure/src/model/RWSDLField.ts
  servicesFilePath(name: string) {
    const ext = this.isTypeScriptProject ? '.ts' : '.js'
    return join(this.pathHelper.api.services, name, name + ext)
  }

expects services to follow the convention:

join(this.pathHelper.api.services, name, name + ext)

so /api/src/services/ the name of the service (as a directory), and the name of the service with ts or js as the extension.

[Sebastian]

Thank you @dthyresson @danny - the “pattern” I used was just a “proof of concept” to check if I could have the files in separated directory structure (ex. to delete all code I wrote if it was . I was checking what did and didn’t work.

I’m OK with doing it the RW way and having managers is a standard, good way to go. The repo has been updated to follow best practices mentioned.

I have also updated the repository and added some documentation with the way to connect the Form to API while using custom Select component (my understanding, hopefully I have not made many mistakes).

I was also able to “fix” the broken Select component - this was indeed GraphQL/API issue as I suspected… mostly.

Getting back on topic… I create a select component that receives an array all books from the Query in EditSeriesCell (how to manage that when I’ll have 1000s of books is another question).
In the Series.books model I keep already assigned books to the series.

I have a request and few questions:

1. Can you or anyone from the team or someone from community check the docs I created as well as the “app” and tell me what I am doing wrong if anything?

2. Apollo adds __typename: 'Book' to the query results and that needs to be stripped before handling it to another component - feels wrong - is there any other patter that should be used to get same (better) results - displaying list of options for “select many” field?

3. How should lists of 100s or 1000s of elements be handled in such select fields?

4. Can anything from that project or docs be added to the RedwoodJS documentation? I am very much willing to help and expand both the code and RW docs to help others get to speed faster with relational data in Forms or using custom components.

Repo link is in earlier post.
Thank you for all the help!

[danny]

Thank you for the reproduction here Sebastian.

Service not implemented
First thing to note, is that this is not an error - it’s a warning coming from the VSCode Redwood extension that is trying to be helpful (but failing in this case!)

As DT pointed out - the VS Code Redwood extension looks for schema’s in a certain place. You are welcome to raise an issue for this - but we are still to make a decision on the ongoing support of this extension.

I assume you are trying this to understand what Redwood is doing underneath rather than actually want to structure your project that way? It doesn’t look like you’re actually implemented a custom sdl - they’re still the same things that would be generated if you used yarn rw g sdl BookSerie ?

Cannot query field “books” on type “BookSerie”. Did you mean “Book”?GraphQL: Validation
This does not seem be an issue in your reproduction - I’m able to make the query - can you double check that it’s still an issue?

exported variable MUST be named just like the query in SDL
This is true, and a requirement. You will thank yourself later for maintaining this practice too!

Imagine having a query getMyBooks and the service function called retrieveBooksFromDatabase - it’s totally fine to do, once you have a few dozen queries and services in your project, you may find it confusing!

Personally I do not think we should add support for renaming the service functions to not match the query - because atleast this way it’s clear that there is a naming convention that must be followed.

I just now regenerated user with SDL and it would add UserResolvers hat were not there a sec ago at all

This is likely that you’ve changed your User model - before it did not have relations, and now it does.
Redwood will generate “UserResolvers” when it realises that there’s a DB relation between User and something else. I now call these “relation resolvers” - but in reality they are just field resolvers.

Any property defined in the export const User = { customField: () => xxx } will only be called, if the request has a customField in it. This can be a powerful tool to optimise your database queries - for example if customField did heavy DB queries, you don’t really want to fetch it from DB unless the client actually requested it.

Apollo adds __typename: ‘Book’ to the query results and that needs to be stripped before handling it to another component -
I’m not sure why you would need to strip it out? Unless you have a prop called __typename it should be harmless.

It looks to me like you’re trying to pass the result of your query, directly into your mutation. So QueryBook → UpdateBook - which seems a little odd. You probably want to pass the data from the form and not the original Book that you passed in.

How should lists of 100s or 1000s of elements be handled in such select fields?

This feels like a philosophical question - do you really want your user to edit 1000s of fields in one go? Why not separate your fields into different categories that they can edit e.g. Metadata, Author details, etc.

[dthyresson]

I +1 the above from Danny and have asked the Discord channel for thoughts on the docs, form use.

@Sebastian

• How should lists of 100s or 1000s of elements be handled in such select fields?

I think it is best to revisit the UX in this case. And why I don’t love forms. They are great for … well… forms like you fill out in an office. But, for more complicated flows or choices, another experience is often best.

Plus – books. They have covers and are pretty and can have long names and you want the author, too, to help distinguish one from another.

I’d try a paginated grid list/cardview of covers that maybe picked your book first, and then you fill out the rest?

[Sebastian]

Cannot query field “books” on type “BookSerie”. Did you mean “Book”?GraphQL: Validation
this no longer happens.

I left it out in the code still since I was working on something different (and it finally worked). I will remove it but it was strange as it “went away” on it’s own after rebooting the PC (overnight) and not when I was restarting Redwood Dev server or when I regenerated the types. Either way I could not reproduce so I’ll remove it in near future.

exported variable MUST be named just like the query in SDL
I do not mind conventions I just prefer them explicitly stated but it should be in the docs actually if my memory serves me right. This needs verification.

This is a big one:
Apollo adds __typename: ‘Book’ to the query results and that needs to be stripped before handling it to another component
I need to strip it out since I am adding this object as prop to the form component so that underlying select can receive all the books that are in the DB (hence the question how could I ex. create a dynamic query that would fetch paginated data for select component in case of 1000s of books).

When saving data from the form that __typename also is being sent and since I needed a custom BookInput type to define input for relational books field that __typename field is something API/RW will complain about if I submit the form.

if you comment line 87 in web\src\components\Admin\BookSerie\EditBookSerieCell\EditBookSerieCell.tsx and try to save the form after changes to books you will get an error:

Variable "$input" got invalid value { __typename: "Book", id: "cl7ol71xf0028ycrc1xvdyqzc", title: "The Way of Kings" } at "input.books[0]"; Field "__typename" is not defined by type "BookInput".

How should lists of 100s or 1000s of elements be handled in such select fields?
That’s what I am doing but I expect to have many Book fields and it’s a relation field. You can assign (or reassign) any book to a series so all of them need to be available. I assume components help but from RW perspective how should I approach writing correct query for that?
Is any kind of results pagination supported out of the box?

[danny]

__typename in graphql queries

if you comment line 87 in web\src\components\Admin\BookSerie\EditBookSerieCell\EditBookSerieCell.tsx and try to save the form after changes to books you will get an error:

Hmm I understand - but I think the problem here is that you’re trying to pass the entire book object into your mutation.

To me it feels odd that you’re selecting a book from a list, and sending the entire book in the mutation (without ever updating any of the fields in it). Would it not make more sense to just send the book Id if the purpose is for the user is to just select a number of books to add to the series?

This will likely solve your “large number of fields” problem too - unelss the user is also editing a book at the same time as creating a collection, there shouldn’t be a need to send all the book fields - only the id of the book you want to. When you generate your sdls with the CLI - it helps you avoid these sort of issues - but I’m not sure about this particualr case

Update
I think I know the source of the problem - in your schema BookSerie has a one to many relation to Books. Which is probably why you wrote the SDL to always send full books, instead of just the book id.

@dthyresson any advice on the best way to handle this? Personally I would’ve changed the mutation to make sure we only send book Ids in the input and use connect - something like this:

[Sebastian]

I am actually doing that in the service:

line 27 @ \api\src\services\serieManager\serieManager.ts

export const updateSerieSetBooks: MutationResolvers['updateSerieSetBooks'] = ({
  serieId,
  serieData,
}) => {
  return db.bookSerie.update({
    where: {
      id: serieId,
    },
    data: {
      ...serieData,
      books: {
        set: serieData?.books?.map((book) => {
          return { id: book?.id };
        }),
      },
      // we cannot use code below this since it will invoke error:
      /*
      Argument data.books.set of type BookWhereUniqueInput needs exactly one argument, but you provided id and title. Please choose one. Available args:
      type BookWhereUniqueInput {
        id?: String
        idCode?: String
      }
      but this type is not generated and cannot be used instead of BookInput
      */
      // books: {
      //   set: serieData?.books,
      // },
    },
  });
};

I have to provide a single argument but since I also want to display the data inside Select component (title) I need full object passed to the component itself.