I haven’t tackled this yet, but I already have a domain setup, so I’m not sure I fully understand the issue here. My URLs are api-STAGE.aws.mystartup.com for the API, and and home-STAGE.mystartup.com for the web app. I’m assuming (perhaps incorrectly) that I can just allow everything across *.mystartup.com…?
As for Flightcontrol, I’ll take another look… we are pre-seed so aren’t looking to spend cash at the moment, unless absolutely necessary.
I’m curious how, and if, you are handling Auth + CORS for your setup. I’m imagining you are using the dbAuth-specific API function and implemented your own integration for the AWS Amplify app.
Because we’ve been having challenges navigating CORS with dbAuth, could you share snippets and how/where you had to manage CORS? (e.g. serverless config file, dbAuth function, GraphQL, etc.)
Yes, this is the first reason why we decided to do our own deployment for frontend side. We didn’t want to deal with CORS, because it’s an overhead of engineering but it’s also a runtime overhead. So we moved to AWS Amplify(that uses CloudFront too). Now we use a rewrite rule in the Amplify app (like a Nginx reverse proxy rule) that rewrites app.domain/api to the API Gateway address. No CORS headers, no preflight request.
@joconor 's last message about serverless stages remembered me of that and another advantages we considered when choosing AWS and serverless.
After test almost every deployment alternative, we realized serverless was one of the less invasive, doing few changes in the codebase after setup. I mean, it’s easy to remove and use another one.
But the most is that we can only deploy. We don’t need to give access to our codebase, we can tag and release our codebase to achieve stability without duplicate it in different branches. We have total control of our build environment and codebase using trunk-based development, tags, and CI/CD.
I don’t know what went in to deciding the current Redwood Serverless deploy architecture. So, I’m asking! Specifically, I’m curious if, since serverless-lift is already used for the front-end deploy of a Redwood app, was the serverless-lift ‘server-side-website’ construct considered? The described AWS architecture of that seems to be pretty much exactly what a Redwood app wants to be:
I’m tempted to see if I can get this working, but if the Redwood team already considered and rejected this, I’d rather not waste my time.
Also, if anyone else has hammered Serverless Framework into a useful state for deploying Redwood apps and avoiding CORS, I’d really like to hear from you. I see @gilliard has gone the Amplify route, but I haven’t quite given up on Serverless.
Lift was “incomplete” at the time we finished what is the current Serverless AWS deploy config. About a month after we finished it, the individuals we were working with at Serverless told us about the new SPA capability (which I think was in Beta at that time). They said it would be a much better fit for Redwood — we just never had a chance to try it:
You should absolutely give it a go if you have the time!
Current CI Deploy Project
I re-added the directory with the Serverless AWS deploy project. You should be able to see all the config and CORS settings we used.
We can try to use this project to test anything you get working. E.g. if you figure out Lift for SPA deployment, creata a PR and let’s see if we can get it going (again).
I’m really not sure what the difference is between serverless-lift ‘static-website’ and ‘single-page-app’, in that both constructs have an AWS architecture of a CloudFront distribution fronting just an S3 bucket. The documentation for both is virtually identical.
If we agree that the goal is to eliminate the need for CORS (and third-party cookies in the case of dbAuth), then we need an AWS architecture that puts the CloudFront distribution in front of both the S3 bucket and the API Gateway to the Lambdas. The serverless-lift ‘server-side-website’ construct does exactly that. The serverless-lift documented example assumes enumerated paths for the assets in the S3 bucket and everything else goes to API Gateway, but I think it is configurable so that I can do essentially the opposite - enumerated paths (/api/* perhaps?) go to API Gateway and everything else goes to the S3 bucket. I don’t know if there are any other “gotchas” that I might run into.
FWIW, my gut tells me that you’re most likely right about Serverless not being the ‘best’ way to support deploying to AWS Lambda. I do think AWS CDK is probably the best long term solution. I’m hoping that serverless-lift server-side-website might be an interim solution until we can invest the time in a more elegant solution, likely based on AWS CDK.
I’d go with the single-page-app just to be safe. Although it would be great to eliminate the need for CORS, I don’t believe that will be possible here. We just need to resolve the issues we’re having with CORS + Serverless deploy.
Also, Redwood does not have server rendering capability (yet). So the server-side-website most likley won’t work. But, hey, maybe you’re seeing a path to get it going! Vercel and Netlify proxy AWS Lambdas to root.domain/api and root.domain/.functions — seems like we should be able to do something similar.
@thedavid I’d like to understand what issues you’re seeing with CORS and your (CI/CD?) deploy. It’s definitely rough, but it works more or less following the published instructions. I did make a WEB_URL env var to avoid editing code for the CloudFront URL, and it does take an additional deploy to get that URL in the deployed configuration. So yeah, not great, but doable.
But! IMO, the bigger problem is the need for third-party cookies in this two domain configuration when using dbAuth. Perhaps my set of constraints is somewhat unique (–WebAuthn a tentpole feature, so need dbAuth; –Need/want ephemeral/semi-stable deploys for testing, etc, so keeping all the AWS configuration together is a big plus). I can make the CORS thing work, even though it’s less than optimal. But the dbAuth third-party cookie issue is fatal for me. I can’t require end users to configure their browsers to allow third-party cookies.
Given that serverless-lift static-website and single-page-app are documented virtually identically, what would single-page-app do differently that would help with problems you’re seeing?
I’m sure there’s a lot I’m simply ignorant about, but I’m not sure what the significance would be whether Redwood is rendering server-side or not (as long as the response can be JSON). It’s just a Lambda function behind API Gateway behind a CloudFront distribution. Of course, I know AWS can be a bit fiddly. There may very well be things about how CloudFront functions get configured or other details that the server-side-website doesn’t give sufficient configurability to allow this to work.
In keeping with my thought that serverless-lift may be an ok interim solution on the way to AWS CDK, I’ve also been wondering how difficult it might be, if the server-side-website construct doesn’t work out, to create a new serverless-lift construct based on server-side-website but tailored to Redwood. I don’t intend to pursue this unless and until things don’t pan out with server-side-website though. Keep in mind that serverless-lift is itself using AWS CDK internally, so it might a good way to get one’s feet wet with AWS CDK
Dropping Serverless support is a really bad news to me.
Why Serverless: is it about cost or design requirement or …?
I want to deploy to AWS in an easy way. Serverless is probably one of the most popular solutions to do that and AWS is probably the most popular Cloud provider.
RW version : 4.4
Did you add any complexity to your Serverless deploy config?
What is your DB provider? (e.g. AWS RDS…)
AWS RDS Postgres
Do you have Auth? If so what’s your setup?
Auth0 for production.
If continue with this deprecation plan I will be forced to stop upgrading RW until find an easy way to deploy in AWS. Vercel, Netlify are not options for my organization.
For @janaka , or anyone else who wants to follow along, here’s where I’m headed:
Having multiple deployment options is a huge advantage for RedwoodJS.
I believe Serverless Framework is the “best” approach for direct AWS Lambda deployments (“best” in quotes because it is very subjective). Why?
RedwoodJS positions itself as the framework for startups. And startups usually have to be very cautious with costs. This means keeping a close eye on adopting services that incur costs.
AWS has a very generous free tier for AWS Lambda, as well as for related services. It’s advantageous for startups to stay within the free tier and limit use to services that do have a free tier.
Serverless Framework does not incur a cost
Why not use AWS CDK?
I think CDK may be the eventual answer, but at this time I believe it’s a pretty big leap to adopt CDK directly. By staying with Serverless Framework (for now) I believe a trouble-free AWS Lambda deployment can be created that will also serve as a stepping stone to direct adoption of CDK
What’s the problem with today’s implementation of Serverless Framework deployments?
In a word, CORS. The current architecture deploys the api side under one domain (within AWS API Gateway) and the web side in a different domain (within AWS CloudFront CDN). This results in an almost “circular dependency” where the web side needs to know the API side domain (already handled today, but a bit awkward with the --first-run flag) and the api side needs to know the web side domain info to properly configure CORS (not handled today at all).
If you’re using dbAuth, the separate domains for web & api also mean that third-party (tracking) cookies need to be enabled in the browser.
Ok, I see the problem, so what’s the answer?
I’m working on creating a new plugin for the Serverless Framework that is tailor made for deploying RedwoodJS applications to AWS Lambda, with API Gateway and a CloudFront CDN in front of both web & api sides
It’s my goal to make this solution sufficiently compelling that the RedwoodJS core team will want to take it over. But if this is just a stepping stone on the way to an AWS CDK based solution without the Serverless Framework, that’d be good too.
What about the serverless-lift ‘single-page-app’ construct?
Unfortunately, that is semantically equivalent to the ‘static-website’ construct that RedwoodJS uses today. Instead of a public S3 bucket, it uses aCloudFront function to redirect accesses that aren’t to certain file types (i.e., static assets) to index.html to allow the client side to handle routing. It does nothing for the api side and would result in a two domain deployment that is the cause of the current issues.
That plugin looks promising, but it doesn’t appear that it is very actively maintained. I haven’t tested it with RedwoodJS. If you feel inclined to test it out I’d be very interested in your results.
AWS has a lot of rough edges that can take a lot of time & experience to learn how to work with. I have some experience with some of those AWS rough edges, but certainly not all of them. I’d welcome anyone who’d like to work on this with me!
We also depend on serverless deploy and been using it for nearly a year, this is bad news and will be a blow for us too, will slow down and put risk in our RW upgrade, even without dropping, the fact that you will not guarantee is a risk.
We are using Lambda 18.x Runtime with Redwood 4.5 successfully, the main justification regarding “The Serverless Framework only supports deploying to AWS Lambdas using Node.js version 14” doesn’t apply and justify this deprecation?
Why Serverless: It’s the most mature and popular servelsss deploy tool for AWS and the one built in to RW.
RW version : 4.5
Did you add any complexity to your Serverless deploy config? Added EventBridge config for one of our functions and API Gateway cors config
What is your DB provider? (e.g. AWS RDS…)
AWS Aurora Postgres
Do you have Auth? If so what’s your setup?
Cognito using custom auth .
Good to hear, @joconor . I agree with your points.
I think there might be a shortcut. When we were creating our deployment many months ago, I didn’t manage to configure CloudFront as a reverse proxy. I read many articles but without success, always getting CloudFront errors without enough information to debug them. Then I decided to move to Amplify, that’s a pretty cool service, and we are fine with it.
But now I saw a new article saying how to do CloudFront reverse proxy using serverless:
RedwoodJS config already have the API_URL env. So theoretically it’s possible to keep CORS disabled in the backend and include the endpoint /<API_URL> in the frontend distribution. Frontend still need to know backend url but the not the opposite.
@gilliard , thanks for pointing out that Dev Community post! It definitely looks like it has some useful information.
I’m still not sure how to knit everything together. The post shows how to create a CloudFront Distribution in front of an API Gateway, but I’m not sure how to handle the S3 part that we need. I.e., does it make sense to still use serverless-lift to get the assets uploaded to S3? If so, then serverless-lift has created the CloudFront Distribution and how do I reference that as opposed to creating a new one? If I don’t use serverless-lift to upload the assets to S3, how do I get them there?
I definitely think it’s worth exploring if & how to set up CloudFront in front of both S3 & API Gateway using existing Serverless mechanisms such as the resources section. I just don’t yet have the knowledge how how to put all the pieces together without getting down into coding up a CDK based solution.
This absolutely introduces circular dependencies, which seems to be a sticking point for @joconor – but it doesn’t really bother me personally.
I rarely setup new environments, and if I need to add a new one, I keep documentation on the steps needed to get it up and running. It requires multiple initial deploys, along with generating certs, updating DNS via GoDaddy manually, updating .env.example files, updating environment variables for the lambdas, manually linking the domain via API Gateway, etc… so not a quick automated process.
If your WebURL is stable, then the circular dependency isn’t really a problem. But if you’re relying on CloudFront generated URLs and routinely removing a deployment and redeploying with the --first-run flag, then the circular dependency can be problematic.