After generating several scaffolds for our app, I noticed that it generates very little testing and no stories. The only test that gets generated is for the api service. I know when you generate the individual items, they generate both stories and tests.
My issue is that this sets an undesirable pattern for those using the scaffolding generator.
Short term: Is there a way to backfill these tests without manually creating them?
Long term: Is there a plan to have the scaffolding create these missing items?
I can think of two ways to fix this for the long term:
Add the missing files into the scaffolding templates and generation system.
Change the scaffolding system to use the individual generator templates to generate scaffolding.
Personally I like option #2, but I would assume that is much more work. It would be nice so the templates aren’t duplicated and exist in one place. It would also be nice when a developer overrides the built in templates with their own.
We had several discussions about this when we first started integrating Jest and Storybook. Our thought was that most folks will end up customizing or throwing away the scaffolded pages as they build out their app. Several of us on the core team are seasoned Rails developers (the framework that inspired the scaffold generators) and none of us had used the Rails scaffold generators in years. It’s very impressive for new developers, or when rapidly prototyping something, but once you’re used to the framework and know what you’re doing they seemed much less useful.
If you do keep them, they’ll almost certainly be customized at some point, and it won’t take hardly any changes before the test suite and/or the generated stories completely break, which can frustrating or scary for a new dev.
If you know you’re going to keep generating scaffolds and keep them around, you can customize the generated templates, including the test and stories files, so that any future scaffolds you create will have those files fully built out. But I don’t think we’re going to make that the default in the framework in the near future.
I too am a long term Rails developer and rarely used scaffolding, mostly because the applications I was working on were all very custom. That said scaffolding works well as a good starting spot for admin CRUD, which is what I was currently doing with it. (Especially since there isn’t an ActiveAdmin/Administrate type gem that I know of)
While I agree that having the generated stories/tests breaking can be scary for new devs, I think not having them and not having to worry about them establishes a poor pattern. From my perspective, one of the first places a change should occur is in the test. Then refining the story to make sure it plays nicely with the changes.
I guess it really depends on what your target audience needs. If it is for quickly prototyping or new developers, then refining the scaffolding makes sense to me. But I understand only so much time and effort to spend on various things.
Are you against adding tests/stories to the scaffolding? I may be up for contributing if it would be of interest. Otherwise I can create custom templates as you said.
That fact that you’re willing to create a PR is definitely helping! I’ll bring it up at the next core team meeting and let you know what everyone thinks!
Normally there’s one every Tuesday but I think we’re going to skip the next one as several folks are out of the office, but the one after that should be on.
The generators actually do create a test and story file – at least yarn rw g component <name> does (unless we changed this recently).
Perhaps you are referring to a different command?
The generated tests and stories are a nice starter but we actually deleted the stub tests since they are a simple render test. You can actually get these render tests from your stories with the storybook addon interaction.
module.exports = {
features: { modernInlineRender: true, interactionsDebugger: true },
// Storybook essentials are baked into redwood
addons: ['@storybook/addon-interactions'],
}
(side note: Michael asked this same question when he first started using Redwood, so I nudged him to see what he thought now that he’s been building for a while…)
@MichaelrMentele This question is specifically for the yarn rw g scaffold [model] command (where you get all the API/UI CRUD).
If I recall correctly, when you first started using Redwood you asked the same question. But I’m wondering what you think now (or if you even use Scaffold generators anymore).
I’ll have to look, but did a paginated sortable/filterable/searchable data grid ever get bundled into scaffolds? All the boilerplate that goes into writing that for admin pages is a massive time sink. I’m finally to the point in a project that I’m doubling back to throw time into Redwood and that was one of the things I remember needing to research further was which feature rich data grid library/component I’d want to use for this task and which ui library to use to render it all. It wasn’t a deal breaker that it wasn’t there day 1, but it would be SO nice to have something I could feed a JSONSpec to or something like that and it just whips me up a new/update form and a rich paginated and/or endless scollable data grid using the spec I provided, which could then be used for any automated testing or documentation. I want that.
Okay @craineum we had a lengthy discussion in our last core team meeting. One concern was the experience of generating 30+ new files and overloading new devs. People are already a little overwhelmed that we create so many as it is, we’re worried that tripling that amount is going to be a lot to handle. We talked about maybe prompting when creating a scaffold, asking if you also want to generate tests and stories, or maybe just compressing the file list somehow so it isn’t so long. Or, update the post-install instructions with a more friendly overview of what was created so you’re not just handed 50 files and being told “good luck!”
There was also a discussion of how much the web side tests are actually used…since all the business logic should be living in your services, api tests seemed more important, and run more frequently. Do you find that’s the case, or are you running the entire suite most of the time?
There was also an idea to just generate tests for the “important” functions, like submitting the form, with good and bad data. Most of the pages that are created just show a cell, so t he pages themselves wouldn’t necessarily need tests. How do you feel about only some files having tests and not others?
Is your concern with missing tests/stories that you actually do keep those scaffolded pages around as admin pages, and want to make sure they don’t break if a DB field is removed, for example? Are you heavily modifying them and want to make sure they stay tested? Are you a completionist and it just feels wrong not to have them? haha
We’re going to shop these same questions around with some teams we’re in contact with that are creating startups using Redwood to hear if/how they’re using scaffolds and how they feel about the lack of tests on the web site.
Sorry this took so long to get an answer that isn’t really an answer yet, but we’re definitely giving it serious consideration!
I have to ask - when using Redwood how do you propose startups develop prototypes - we are currently using scaffolds and creating many to many relationship with custom (react-select) component takes a lot of time (due to a lot of files needing to be changed) and some gotchas.
What is the proposed solution? I will try to provide detailed documentation when possible on our process but having that built in would be a big plus (I need to change the code examples I have in the doc already).
@rob Your concern is partially valid but the amount of files is already huge so adding few more but shaving HOURS of development would be a huge plus. This is especially true when stuff like DB changes - and that requires changes all over the place so regenerating scaffolds is faster than doing this manually.
I would love to see that kind of functionality in the tutorial as it’s a huge part of any app and I have not seen any detailed explanation of m2m interface.
@rob Thank you for the thoughtful time and effort put toward my original issue. I will attempt to better explain my point of view. We (ZEAL) are using the scaffold to build out CRUD pages that we are using as an admin section for a RedwoodJS template application. With the template we are trying to bypass much of the boilerplate that is associated with applications that bogs down startups, and secondarily gives new developers a foundation from which to work from. To give you some background on the request.
We try to unit test things that aren’t tested by some other dependency
i.e. test front/back end logic, yes
test Redwood/React Hook Form, no
Componentize things as much as possible
For reuse
Separate work between dev/design
We test feature/e2e happy path only
This means we are adding the following:
Tests for all backend logic
Tests for all components
Some components are only ‘make sure the component can render’
Validate the data we are expecting
Validate the actions the user can take
Move complex front end logic into hooks and test separately
Stories for components
We are also adding in playwright for e2e testing
We have no problem adding in these stories and tests ourselves, but I see some issues with Redwood scaffold generators not being more robust (outside of ZEAL’s desires for our template):
For new developers, they may not know the benefit of tests/stories and could get a ways down the road before they realize and then could become overwhelmed by going back depending on how much scaffolding they added. (This is of course our biased opinion that most things need tests)
Test describe what they are testing. Having tests is better than having documentation as you can rely on the tests to tell you what the code being tested functionality should be (even if it needs to change because they are failing).
For startups, as @Sebastian just commented, Scaffolding could be much more robust to save them time. This would definitely be above and beyond the stories/tests, but the stories/tests could incorporate more coverage than checking for exceptions to start.
Hopefully with that background, you can understand my position a little better. In response to your email:
Overwhelming the new developer
Yes, generating a whole bunch of files can be overwhelming, but not generating them can be as well. Personally I think the added benefit of them outweigh the negative.
Personally I wouldn’t prompt, but I could see adding an argument to make them not generate.
Adding post install docs or a link to current documentation is a great idea
Front-end testing
I don’t believe all business logic can or will fit into the backend only, at least I have never worked on a project that “pure”. (i.e. because of this value in the data, I render this thing differently)
Outside of that I think user actions are important to test as well, as that is the front-end logic that needs testing.
We always run all tests as part of our CI/PR process.
Creating tests for the “important” bits would be better, it would cover the user action bits. We could argue the benefit of testing expected data (thus requiring tests for most all), but that is something I could see being a good compromise.
I think the scaffold makes for a great admin section.
Selfishly, I would love to use the scaffold as our admin section generator. Builds everything you need with little to no additional coding needed (think Prisma studio). With tests that back it up.
I also loved the idea from @Sebastian that a developer could delete a whole section and rebuild it with the generator and not loose anything of consequence.
I was thinking of moving this into an RFC ticket, then taking on the work to improve the scaffold generators. But I definitely want the core teams take on it. So I greatly appreciate you getting back with me and being so thoughtful in your response. Here are the things I would want to address with scaffolding:
Layout with fixed Call-to-Action, be nice if call to action was dependent on page (I know you have done some work here, but haven’t gotten a chance to review yet)
Handle relationships: Many-to-One, Many-to-Many
Add MetaTag titles to appropriate pages/components
Add tests and stories
Okay, that was a ton. Hopefully it relays some of the drive to address this.
Thank you so much for the thoughtful reply, and thanks for your patience in getting replies from us! We had another core team meeting today and it sounds like we’re leaning towards including them going forward. If you could open an RFC issue and lay out your strategy, that would be awesome. We’d like to also use it as a kind of poll to hear from some other folks in the community to see if they have a strong opinion one way or the other.
Our remaining concerns seem to be centered around:
Overwhelming new users with files (but if they go through the tutorial they learn which ones to focus on and which ones to ignore when you’re just getting started)
Implying that the scaffolds are all you need to build a Redwood app, that they’re fully featured application code that will magically do everything for you. To date we’ve taken the view that they’re like literal scaffolding around a building: hastily built, held together with screws and duct tape, just there temporarily to give you support until the “real” app is in place, then they’re removed. This has been the major blocker to adding support for selecting many-to-many relationships as a fully-featured component (rather than just letting you enter an ID for a foreign key)—the more features we add to them the more people will want and we worry that Redwood just becomes a framework for generating scaffolds!
It would be great if this initial RFC could just focus on the tests/stories (adding MetaTags as well, we should have added those years ago!) and maybe create a separate one for about adding many-to-many relationship support?
New Redwood user here… I just found this thread after banging my head against the wall wondering why the scaffold generator wasn’t creating stories and tests for me. I think because the tutorial and so many of the videos about Redwood out there emphasize the story / test behavior that I just assumed something was broken with scaffolding. At a minimum I think it would be helpful if the tutorial at least called this out as expected behavior, unless I missed it!
i come from crud generator world like rails and cakephp.
IMO, having those extra files while scaffolding helped a lot in learning process. how i understand mvc back then, i keep cruding again and again and again, from there i started to see the relations in model, how every controller::method have their own view, component, behaviour and helper. this is not something i would understand by reading.
maybe can add an option in the scaffold command, like --with stories tests ? for us newbies?
Not that it answers your question, but you do get tests and stories with generated components and cells when they are generated separately from the scaffold.
I’ll bring it up at the next core team meeting and let you know what everyone thinks!
We had another core team meeting today and it sounds like we’re leaning towards including them going forward. If you could open an