Okay I figured it out
This was the fix I needed in serverless.yml
So it was a config issue, not a bug in redwood, but it does have implications for the serverless setup command I think.
