Thanks @sladomic Very helpful.
FYI: I updated my questions:
- what is your DB provider? (e.g. AWS RDS…)
- do you have Auth? If so what’s your setup?
Our CI/CD deploys using dbAuth. Which may by adding to the complexity for this deployment provider.
updated 
- why Serverless:
Since we use AWS and wanted to keep costs down we opted for serverless - what version of Redwood?
4.1.4 - did you add any complexity to your Serverless deploy config?
- Yes, all AWS services except Parameter Store are included in the serverless files
- We also have an additional side for python-related backend tasks which has an additional serverless file
- We use a plugin “serverless-local-schedule” which creates cron jobs with a localized time
- what is your DB provider? AWS RDS Postgres (not serverless at the moment because it isn’t supported yet in our region)
- do you have Auth? If so what’s your setup?
Azure Active Directory (the only thing not on AWS) - anything else that might be helpful?
We useyarn redwood deploy serverless --pack-onlyand thenyarn serverless deploy --stage productionbecause we have some additional steps in between. So if it’s possible to keep the first part somehow, that would be enough for us.
1 Like
Hi @thedavid,
Thanks for continuing this conversation.
- why Serverless: is it about cost or design requirement or …?
- Both cost requirement as well as ability to scale without having to switch providers (and by scale, I’m including the ability to include other AWS services in our deploy). We can start with zero cost while in development due to the generous AWS allowance on Lambda & related services
- what version of Redwood?
- RedwoodJS version 4.4.2 (will do the 4.4.3 update shortly)
- did you add any complexity to your Serverless deploy config?
- Updated NodeJS version without any problem (not seeing the error you report)
- Working on CORS now, don’t have that working yet.
- Have a function defined to receive Google PubSub messages in addition to
graphqlandauth - I anticipate that eventually we will put the Lambda functions as well as an AWS hosted PostgreSQL db within a VPC, but we’re not there yet. The fact that this direction is open to us with Serverless Framework is a big appeal.
- what is your DB provider? (e.g. AWS RDS…)
- Currently PostgreSQL on Railway.
- Need to do our eval of the best approach to AWS hosted PostgreSQL but will be doing some flavor of AWS PostgreSQL
- do you have Auth? If so what’s your setup?
- Yes. dbAuth. Banging my head against the CORS issue right now.
- anything else that might be helpful?
- AWS is definitely the 800lb gorilla in the serverless space. Given Redwood’s mission to enable startups, there’s a natural fit here even despite some technical hurdles. Serverless Framework has made it easier to deploy to AWS, but I’m not completely wedded to Serverless Framework. I’m comfortable with Serverless Framework, in that I’ve worked with it in the past (and that was a huge factor in choosing RedwoodJS). If something like Serverless Stack or directly using the AWS CloudFormation SDK were to work better, I think we could manage that. But deploying directly to AWS without additional cost is a must.
- Our workflow makes heavy use of the
stagefeature of AWS & Serverless
Redwood deploy via Serverless Framework has had numerous rough edges since day 1. I’m not in a position to fix those issues myself, but I would very much like to work with other Redwood users who are currently (or interested in) deploying using Serverless.
There are some issues that go beyond just the deploy, but do impact the deploy, such as Redwood’s use of dotenv and dotenv-defaults that play funny with other things that use those packages.
And clearly, the Serverless deploy hasn’t been touched in some time, as evidenced by the deprecation warnings that deploying via serverless currently gives.
I think if a few of us put our heads together we can likely make serverless deploys work smoothly for rwjs
I’m going to venture a guess that dbAuth and CORS may be the major issues in making a Serverless deploy go smoothly (i.e., not the supported version of NodeJS). I haven’t solved the CORS issue for my application yet, but I’m confident I will. But I do expect it will be a bit janky. Because of our use of multiple stages, we typically work with both the AWS assigned API Gateway URL and the Cloudfront URL. Since the Cloudfront URL will be assigned on first deploy, there’s a circular dependency where the front-end needs the API Gateway URL baked in, but the back-end needs the Cloudfront URL set up for CORS. Yes, a bit messy, but doable.
Our Project
Hello David,
My company uses the Serverless Framework deploy for RedwoodJS, and we will be sad to have to see it go. We would love to see if we can help support in the future.
- why Serverless:
- We use Serverless because it allowed us to easily get into AWS (our provider) for scaling. Similar to another reply, we also have other services that we deploy using Serverless including AWS Fargate containers. We wanted the ability to tweak the underlying Lambdas, and loved Serverless’ claim that we could deploy to other providers including Azure.
- what version of Redwood?
- Currently, 4.0.1
- did you add any complexity to your Serverless deploy config?
- Web: no.
- API: yes, we have an AWS S3 bucket which holds some certs we use to connect to a Temporal instance. Also, envs + cors.
- what is your DB provider? (e.g. AWS RDS…)
- AWS Postgres RDS
- do you have Auth? If so what’s your setup?
- Yes, we use Azure SSO. No problems here.
- anything else that might be helpful?
- We like the flexibility of Serverless to tweak the underlying Lambda.
Take a look at what we had to do in the CI/CD deploy project:
^^ You can explore that commit but might be better off cloning and reverting to explore.
There is config you need to add in the Serverless config file. As well as Redwood API (Auth function, GraphQL, etc).
Tricky indeed.
Keep the use-cases coming, everyone! Very helpful.
Worst case, we will not remove Serverless deploy but instead deprecate it. In practicality this will mean:
- we won’t be running CI/CD; so no guarantees
- we’ll indicate
rw setup aws-serverlessis deprecated - we’ll keep the
rw deploy ...intact
I think the future here is using AWS CDK. @peterp has explored this and is very optimistic. And Serverless (company) has support although they don’t advertise it very well — “Serverless Lift”:
Note the “Single page app” section further down the page.
Still just processing and reacting as we go. Any thoughts or comments to the above?
1 Like
I’ve been lurking in this thread for a while now. I just switched to writing a new project in Redwood and the Lambda possibility was one of the selling points. Vercel and Netlify are great tools but when building for clients it’s often easiest to just hand them over a single AWS or GCP account where all resources are hosted. AWS Fargate is an option but it’s preferable to run serverless for some services that have high usage during the limited time during the day and it also makes branch previews very easy to set up.
As you and @peterp are looking into AWS CDK, I suggest looking at Pulumi at the same time if you haven’t already. That way it might be possible to have some shared code/plugins for running on GCP or Azure too. With Pulumi I don’t think there is any need to use the Serverless platform at all.
Adding Pulumi could be an infra extension to Redwood allowing an opinionated way to handle your resources and build it into the deploy step. Essentially allowing users to handle resources such as crons, queues, db setup and more as part of existing CI flows.
Hi @thedavid,
I would definitely suggest a deprecation period. And I would recommend to the RedwoodJS team that you consider adopting some sort of official policy of deprecation. I think this would go a long way toward giving adopters confidence that Redwood is a stable platform to build on.
Regarding adopting AWS CDK as the way forward, that does sound promising. AWS CDK didn’t exist when I first used Serverless, and Serverless has served my limited needs to date so I haven’t had reason to get my hands dirty with AWS CDK. I’m certainly willing to!
The RedwoodJS Serverless deploy is already using “Serverless Lift” for the frontend deploy. If Serverless Lift makes sense as the path to adopting AWS CDK, I think that makes a deprecation period for the current Serverless implementation even more compelling.
I think it would be helpful (and I’d like to participate in this) to better identify the issues involved in doing a deploy to AWS with dbAuth support. I don’t think there’s any issue with NodeJS versions – @thedavid how can we nail down why you’re seeing a warning/error, but I’m not?.
If we’re going to start recommending CDK we should also consider taking a look at SST, it’s an extension of CDK that makes the DX much nicer.
3 Likes
Hello, @thedavid
We are using serverless deployment here for our api side.
When we decided to bump to Node18, I thought it was straightforward and changed the runtime in the serverless.yml file to nodejs18.x. I also couldn’t find in the serverless docs any reference about supported runtimes.
Now, our tests env already run Node 18. Seems ok.
- why Serverless: is it about cost or design requirement or …?
Cost was important but we decided to have the most control and flexibility as possible. So we choose AWS. - what version of Redwood?
4.2.2 - did you add any complexity to your Serverless deploy config?
Yes. We have a Redis instance for graphql caching. Because of using Redis, we had to create a VPC to put our lambdas. And because of using a VPC, we had to create an Internet Gateway to access the Internet inside a VPC. All this infrastructure is in CloudFormation templates included as resources in the serverless file - what is your DB provider? (e.g. AWS RDS…)
AWS RDS - do you have Auth? If so what’s your setup?
DB Auth - anything else that might be helpful?
We only use for api side. We host our frontend using AWS Amplify.
Update: Now we got production env running Node 18 too.
Update 2: We also use Node 18 in our deployment Github Action
2 Likes
Wow, this is disappointing to hear.
- why Serverless: is it about cost or design requirement or …?
- I want to deploy to AWS. I have many years of experience with it and want to leverage that instead of learning an entirely new platform. Honestly I didn’t think deploying to AWS was some controversial opinion, so it surprises me that Redwood is deprecating the one pathway to do that (afaik?). If AWS wasn’t an option, there is a good chance I would have skipped RedwoodJS.
- what version of Redwood?
- 4.4.3
- did you add any complexity to your Serverless deploy config?
- Yes, quite a bit on the API side. I create VPCs, gateways, and the database using serverless.yml.
- I also have a custom deploy process that does assorted setup, then
yarn rw deploy sls --stage "$STAGE", then cleanup
- what is your DB provider? (e.g. AWS RDS…)
- AWS RDS Postgres
- do you have Auth? If so what’s your setup?
- Not yet, I planned on doing custom auth
If you decide to go through with deprecating it, then I will either not upgrade, or patch your code so that I can continue using my current deploy process.
Hello fellow Redwood Users deploying with Serverless:
I’m hoping to get some clarity from @thedavid on the actual problems eventually. I believe the NodeJS version thing is a red herring. From what I’ve observed, it looks more like the AWS two-part deploy, with front-end and back-end each living in their own domain (CloudFront assigned URL & API Gateway assigned URL, respectively) causes a lot of pain for their CI/CD workflow with dbAuth.
Front-end & back-end being in separate domains introduces CORS issues, as well as third-party cookie issues for dbAuth. Addressing the CORS issue is akin to resolving a circular dependency. Already, the front-end needs to know the back-end URL, but for CORS, the back-end also needs to be configured with the domain of the front-end. The Serverless deploy already has the --first-run flag to handle connecting the front-end to the back-end, but configuring CORS means, at a minimum, doing a deploy, finding out the CloudFront URL, editing code and serverless.yml with the front-end origin then doing another deploy with the code & configuration fixes.
IMO, the third-party cookie issue is worse. I don’t know about the RWJS CI/CD pipeline, but in real-world usage, it’s not practical to require browsers to accept third party cookies. Of course, for production, one would almost certainly put both front & back ends in the same domain. The Serverless Framework doesn’t “just do” this.
Using the Serverless ‘stage’ concept to spin up new environments with minimal fuss is a huge advantage, but the current state of the Serverless deploy configuration leaves too much ‘fuss’ to deal with manually.
I’ve seen a lot of people suggest alternate approaches (Serverless Stack, AWS CDK, etc), but (IMO) until the issues with the Serverless deploy are correctly identified it doesn’t make sense to look at other solutions. If I’ve correctly identified the issues as being CORS & third-party cookie complications arising from front-end & back-end living in separate domains, I don’t believe solutions like SST or CDK will “just do” the necessary work to resolve that. But I do think the issue of putting both front-end & back-end under a single domain is solvable, hopefully with minimal fuss, with any of the AWS deploy solutions, including Serverless Framework.
- do you have Auth? If so what’s your setup?
DB Auth
I’m curious how, and if, you are handling Auth + CORS for your setup. I’m imagining you are using the dbAuth-specific API function and implemented your own integration for the AWS Amplify app.
Because we’ve been having challenges navigating CORS with dbAuth, could you share snippets and how/where you had to manage CORS? (e.g. serverless config file, dbAuth function, GraphQL, etc.)
Thank you @sean-ac Very helpful. But let me clarify a few things:
I want to deploy to AWS. I have many years of experience with it and want to leverage that instead of learning an entirely new platform. Honestly I didn’t think deploying to AWS was some controversial opinion, so it surprises me that Redwood is deprecating the one pathway to do that (afaik?). If AWS wasn’t an option, there is a good chance I would have skipped RedwoodJS.
^^ Most startups (and they are many) deploy to AWS directly. They just set it up themselves. I (we) think AWS is a great option. I just don’t think Serverless Framework is the best option for Redwood. And, admittedly, neither did the people we worked with at Serverless when we created it over a year ago — there just weren’t other/better deploy-to-lambdas orchestration services availabe so we gave it a go.
Frankly, my favorite deploy-to-AWS service right now is Flightcontrol, which deploys to Fargate. But there are others in the hopper.
Not yet, I planned on doing custom auth
^^ Keep us posted as this is where you might run into challenges with CORS
If you decide to go through with deprecating it, then I will either not upgrade, or patch your code so that I can continue using my current deploy process.
^^ Deprecation != dropping. (See above dialogue about no longer dropping and what Deprecation would mean.) Nothing would change on your end being able to run yarn rw deploy serverless.
I haven’t tackled this yet, but I already have a domain setup, so I’m not sure I fully understand the issue here. My URLs are api-STAGE.aws.mystartup.com for the API, and and home-STAGE.mystartup.com for the web app. I’m assuming (perhaps incorrectly) that I can just allow everything across *.mystartup.com…?
As for Flightcontrol, I’ll take another look… we are pre-seed so aren’t looking to spend cash at the moment, unless absolutely necessary.
1 Like
I’m curious how, and if, you are handling Auth + CORS for your setup. I’m imagining you are using the dbAuth-specific API function and implemented your own integration for the AWS Amplify app.
Because we’ve been having challenges navigating CORS with dbAuth, could you share snippets and how/where you had to manage CORS? (e.g. serverless config file, dbAuth function, GraphQL, etc.)
Yes, this is the first reason why we decided to do our own deployment for frontend side. We didn’t want to deal with CORS, because it’s an overhead of engineering but it’s also a runtime overhead. So we moved to AWS Amplify(that uses CloudFront too). Now we use a rewrite rule in the Amplify app (like a Nginx reverse proxy rule) that rewrites app.domain/api to the API Gateway address. No CORS headers, no preflight request.
1 Like
@joconor 's last message about serverless stages remembered me of that and another advantages we considered when choosing AWS and serverless.
After test almost every deployment alternative, we realized serverless was one of the less invasive, doing few changes in the codebase after setup. I mean, it’s easy to remove and use another one.
But the most is that we can only deploy. We don’t need to give access to our codebase, we can tag and release our codebase to achieve stability without duplicate it in different branches. We have total control of our build environment and codebase using trunk-based development, tags, and CI/CD.
2 Likes
Open question to Redwood core team:
I don’t know what went in to deciding the current Redwood Serverless deploy architecture. So, I’m asking! Specifically, I’m curious if, since serverless-lift is already used for the front-end deploy of a Redwood app, was the serverless-lift ‘server-side-website’ construct considered? The described AWS architecture of that seems to be pretty much exactly what a Redwood app wants to be:
I’m tempted to see if I can get this working, but if the Redwood team already considered and rejected this, I’d rather not waste my time.
Also, if anyone else has hammered Serverless Framework into a useful state for deploying Redwood apps and avoiding CORS, I’d really like to hear from you. I see @gilliard has gone the Amplify route, but I haven’t quite given up on Serverless.
1 Like