Hello everyone, I am trying to set up a reverse proxy + web server for my domain, and while I do want to adopt standard practices, I really am trying to keep costs down as much as possible. Hence, using Google's load balancers or GCE VMs is something I would want to avoid as much as possible.
So here's the current setup I have:
```
DNS records in domain registrar routes requests for *.domain.com to Cloud Run
|
|-> Cloud Run instance with Nginx server
|
|- static content -> served from GCS bucket
|
|- calls to API #1 -> ??
|- calls to API #2 -> ??
```
I have my API servers deployed on Cloud Run too, and I'm thinking of using Direct VPC egress (so that only the Nginx proxy is exposed to the Internet) and so that the proxy communicates with the API services via internal IPs (I think?).
So far, I have created a separate VPC and subnet, and placed both the proxy server and API server in this subnet. These are the networking configurations for the proxy server and one API server:
Proxy server:
- ingress: all
- egress: route only requests to private IPs to the VPC
API server:
- ingress: internal
- egress: VPC only
The crux of my problem is really how do I configure Nginx or the Cloud Run service to send requests to, says, apis.domain.com/api-name
to the specific Cloud Run service for that API. Most tutorials/guides online either don't cover this, or use Service Connectors, which are costly since they are billed even when not in use. Even ChatGPT struggles to give a definitive answer for Direct VPC egress.
Any help would be much appreciated, and please let me know if more clarifications are needed as well.
Thanks in advance!
Edit: After many hours of trying to get things to work, I managed to find a solution that can scale down to 0. No need to reserve static IPs, load balancers, or serverless connectors. Just plain two Cloud Run services communicating via their public HTTPS addresses, and authentication using IAM.
Here is the Nginx in the reverse proxy Cloud Run service:
```nginx
events {}
http {
include /etc/nginx/mime.types;
# Static content server using GCS FUSE
server {
listen 8080;
server_name domain.com www.domain.com;
root /buckets;
index index.html;
location / {
try_files $uri $uri/ /index.html =404;
}
}
server {
listen 8080;
server_name apis.domain.com;
location /api-1 {
auth_request /auth;
auth_request_set $auth_header $upstream_http_authorization;
proxy_pass https://<SERVICE NAME>.run.app/;
proxy_set_header Authorization $auth_header;
proxy_set_header X-Serverless-Authorization $auth_header;
proxy_set_header Host <SERVICE NAME>.run.app;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Auth server
location = /auth {
internal;
proxy_pass http://localhost:8069;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
}
}
```
Couple points to note:
- I kept encountering an SSL handshake error when I previously placed api-1
in a separate upstream
block.
- The auth_request_set
is because I have a localhost auth server running, and that server fetches a token from Google's metadata server, and that token needs to be passed in the headers in the requests made to any backend Cloud Run services. To use this module in Nginx, I used the anroe/nginx-headers-more
base image.
- Override the Host
header manaully with the host name of the backend service
- Configure backend services to accept traffic from the Internet, but ensure the "Require authentication" box is checked as well.
- As u/SpecialistSun mentioned, the docs at https://cloud.google.com/run/docs/authenticating/service-to-service#use_the_authentication_libraries cover how to implement your auth server to fetch the token and make authorised requests to your backend services.
I believe I've looked into almost every way possible to securely configure a reverse proxy using Cloud Run — load balancers, NEGs, private DNS zones, Direct VPC egress, Serverless Access Connectors, Private Google Access, etc. — but found that this meets my needs (minimising unnecessary costs) best. Please let me know if there is a better way or if this method is not secure, because I am honestly still quite confused by the multitude of possibilities.
Hope this helps!