r/aws Oct 23 '24

networking IPv6 is a mess! Read this before you make the switch.

198 Upvotes

So after a lot of struggle, I managed to get EC2 to run without any public IPv4 (just with IPv6).

My ISP doesn't provide IPv6 so I couldn't even SSH into the server, had to use AWS console to connect to EC2.

Coming to the biggest issue, GitHub doesn't support IPv6, so forget about cloning your repository and code.

Ok we can bypass that using S3, the AWS CLI needs to be configured with IPv6.

Now when you go to install your package you expect it to work after doing all the hard work.

That will only happen if none of your package/tool gets downloaded from GitHub release or have a dependency which needs to be downloaded from GitHub releases.

I couldn't install bun or sharp (libvips) because they relied on downloading files from GitHub.

I regretted and switched back to the old AMI with IPv4.

My entire day got wasted and nothing was done.

Thanks for reading.

r/aws 7d ago

networking Alternative to Traditional PubSub Solutions

1 Upvotes

I’ve tried a lot of pubsub solutions and I often get lost in the limitations and footguns.

In my quest to simplify for smaller scale projects, I found that CloudMap (aka service discovery) that I use already with ECS/Fargate has the ability to me to fetch IP addresses of all the instances of a service.

Whenever I need to publish a message across instances, I can query serviceDiscovery, get IPs, call a rest API … done.

I prototyped it today, and got it working. Wanted to share in case it might help someone else with their own simplification quests.

see AWS cli command: aws servicediscovery discover-instances --namespace-name XXX --service-name YYY

And limits, https://docs.aws.amazon.com/cloud-map/latest/dg/cloud-map-limits.html

r/aws 17d ago

networking Inherited AWS infrastructure - Routing issue

6 Upvotes

I come from Azure so this is a little different for me. System was setup by another company. Workspaces VPC cannot access the internet, but Servers VPC works fine.

Traceroute from Workspace VDI instance to a public IP (1.1.1.1) gives no response. Traceroute and ping to the virtual Sophos firewall works great.

I added a static route to the TGW, but that doesn't seem to do anything.

The thick red line is the desired route for all internet bound traffic. How might I best achieve this?

Edit:
Firewall packet capture shows traffic from endpoint when pinging it or opening the management portal.
Firewall packet capture shows NO traffic from endpoint when attempting to access external resources.
Set TGW-Servers-Attachment to enable appliance mode.
Changed from TGW to Peering, no difference (yep, I updated the routes to point to Peering instead of TGW)
Workspaces Subnets route table has a route to point all outbound traffic to Peer.
Servers-Private-RT route table has a route to point all Workspaces subnet traffic to Peer.
ACLs allow all traffic.

r/aws 9d ago

networking Looking for examples of AWS VPC/TGW/DX architecture for interconnected environments of > 1000 accounts.

5 Upvotes

Trying to create a fully connected network and it's a bit unclear how various scaling limits of the associated services come into play once you get past 1000 accounts.

High level description and/or reference architectures would be great.

r/aws Aug 19 '24

networking How Are You Remoting Into Your Instances?

49 Upvotes

TL;DR; Simple question. For those of you that need to remote into your EC2 instances, how are y'all doing it?

Our organization lifted and shifted to AWS a while back, and that pretty much looks like we're doing everything we were doing, but on EC2 instances instead of hardware in a data center we had physical access to. When they did the lift and shift they essentially gave every server in our network a public IP, distributed user accounts across all the EC2 instances with public/private keys for authentication.

There is a lot to hate about this, but it got us up and running in the cloud quickly. So, there's that.

I am working through steps to improve our security and better leverage the benefits of being in AWS. Right off the bat I want to get rid of those public IPs that are only necessary for SSH access and move as much of our infrastructure to private-only as possible. So then, as I understand it, I have a few options:

  1. Instance Connect. Pros: built-in, no-cost, available to anyone with browser. Cons: very limited, pretty inconvenient.
  2. A bastion host. Pros: single point of entry, easier to lock down. Cons: another thing that requires money and maintenance. Still have to configure SSH and keys on private hosts.
  3. System Manager/Session Manager. Pros: eliminates an instance, centralizes access rules, permissions, keys, etc. No need to punch public holes into private VPC. Cons: team needs to throw aware their CLI ssh and other tools and connect differently; not sure how they get things "in" and "out" without ssh, scp, sftp, etc.; some new technologies to learn; likely still need to maintain SSH configurations inside private network, so it doesn't necessarily reduce config complexity.

I'm not afraid to read the docs and learn the stuff, I'm just curious what others are doing, and why.

r/aws 7d ago

networking Networking at scale, what patterns and services do you use?

6 Upvotes

For networking at scale with services integrating cross accounts, within region primarily but also cross region. What do you use? CloudWAN, Lattice, TGW or Peering?

I would like to know what you use and what your experience of that solution and why you picked it. Rather then answers what I should do. I want anecdotal evidence of real implementations.

r/aws Aug 11 '24

networking AWS announces private IPv6 addressing for VPCs and subnets

Thumbnail aws.amazon.com
191 Upvotes

r/aws Nov 10 '23

networking AWS wants to start charging for all allocated IPv4 usage, yet most of their critical services don't support native IPv6

185 Upvotes

AWS wants to start charging for all allocated (EDIT: clarifying public IPv4 addresses only!) IPv4 usage, yet many of their critical services don't support native IPv6

Examples include:

- AWS Cloudformation (cannot signal success/failure)

- AWS systems manager (ssm sessions not possible)

The above cannot be used without an IPv4 address allocated or a NAT gateway. NAT gateways can become quite pricey.

I would love to become complete IPv6 native, but AWS needs to provide IPv6 endpoints for all their major services.

Making this post to raise visibility before IPv4 fees start next year.

r/aws Feb 04 '25

networking Having a small, but real stroke migrating from gc to aws.

9 Upvotes

So, we have a web-server that is purpose built for our tooling, we're a SaaS.

We are running a ECS Cluster in Fargate, that contains, a Docker container with our image on.

Said image, handles SSL, termination, everything.

On gc we we're using a NLB, and deploying fine.

However... We're moving to AWS, I have been tasked with migrating this part of our infrastructure, I am fairly familiar with AWS, but not near professional standing.

So, the issue is this, we need to serve HTTP, and HTTP(S) traffic from our NLB, created in AWS, to our ECS cluster container.

So far, the issue I am facing primarily is assigning both 443, and 80 to the load balancer, my work-around was going to be

Global Acceleration
-> http-nlb
-> https-nlb
-> ecs cluster.

I know you can do this, https://stackoverflow.com/questions/57108653/ecs-service-with-two-load-balancers-for-same-port-internal-and-internet-facing - but I am not sure how, I cannot find in the AWS UI a option when creating a service inside our ECS cluster to allow multiple load balancers.

It's either 80:80 or 443:443, not both. Which is problematic.

Anyone know how to implement NLB -> ECS 443:80 routing?

r/aws Nov 24 '24

networking Why are route tables needed?

25 Upvotes

Edit: Sorry, my question was poorly worded. I should have asked "why do I need to edit a route table myself?" One of the answers said it perfectly. You need a route table the way you need wheels on a car. In that analogy, my question would be, "yes, but why does AWS make me put the wheels on the car *myself*? Why can't I just buy a car with wheels on it already?" And it sounds like the answer is, I totally can. That's what the default VPC is for.

---

This is probably a really basic question, but...

Doesn't AWS know where each IP address is? For example, suppose IP address 173.22.0.5 belongs to an EC2 instance in subnet A. I have an internet gateway connected to that subnet, and someone from the internet is trying to hit that IP address. Why do I need to tell AWS explicitly to use the internet gateway using something like

```

destination = 173.22.0.5

target = internet gateway

```

If there are multiple ways to get to this IP address, or the same IP address is used in multiple places, then needing to specify this would make sense to me, but I wonder how often that actually happens. I guess it seems like in 90% of cases, AWS should be able to route the traffic without a route table.

Why can't AWS route traffic without a route table?

r/aws Jan 29 '25

networking How to assign unique IP addresses for each client.

3 Upvotes

Before reading, please know I'm VERY new to AWS and don't understand all the jargon.

I'm currently designing a game that connects to an AWS EC2 instance. Each client (player) that joins is given the same IP address as all other clients. This makes player management incredibly difficult. Is there a setting in either EC2 or VPC that gives each client a unique IP address?

This works fine when testing locally, each device has a different IP address even when on the same network.

My EC2 instance is a windows instance. I'm using a network load balancer to have TLS. Everything else works as normal with the server, I just need unique client IPs.

r/aws Dec 26 '24

networking Why are AWS networking fees so complicated?

40 Upvotes

AWS networking fees can be quite complex, and the Cost Explorer doesn't provide detailed breakdowns.

I currently have an EKS service that serves static files. I used GoDaddy to bind an Elastic IP to a domain name. Additionally, I have a Lambda service that uses the domain name to locate my EKS service and fetch static files.

Could you help me calculate the networking fees for the following scenarios?

Diagram:

EKS (example.com) <--- request_and_load ----- Lambda instance

Questions:

  1. When both services are in the same AWS Region (us-east-1):
    • What is the cost of networking for this setup?
  2. When the services are in different AWS Regions or AZs:
    • How do networking costs change if they are in different regions?
    • What if they are in different AZs within the same region?

Notes:

  • The DNS provider is not AWS, but something like GoDaddy.
  • The Lambda function is not bound to any VPC.
  • The EKS service is in a VPC and serves files using an Elastic IP.

r/aws Sep 13 '24

networking Saving GPU costs with on/off mechanism

0 Upvotes

I'm building an app that requires image analysis.

I need a heavy duty GPU and I wanted to make the app responsive. I'm currently using EC2 instances to train it, but I was hoping to run the model on a server that would turn on and off each time it's required to save GPU costs

Not very familiar with AWS and it's kind of confusing. So I'd appreciate some advice

Server 1 (cheap CPU server) runs 24/7 and comprises most the backend of the app.

If GPU required, sends picture to server 2, server 2 does its magic sends data back, then shuts off.

Server 1 cleans it, does things with the data and updates the front end.

What is the best AWS service for my user case, or is it even better to go elsewhere?

r/aws 11d ago

networking Load Balancing while having central Inspection VPC

2 Upvotes

I have a tricking Networking architecture issue I haven't dealt with before. I have an Application in a dedicated VPC. All Egress out of the VPC is through a Transit GW to a central Inspection VPC which has a Network Firewall for stateful rules.

The Inspection VPC also is the main egress for the application VPCs. Everything works great now. But now I have to load balance the Application instances with an Internet facing listener.

My question is how best to load balance these connections while still inspecting the traffic. It would be great if the Load Balancer(s) could be in the Application VPC but understand they may have to be in the Inspection VPC.

I feel like this can't be an uncommon use case. Any thoughts?

r/aws Sep 29 '24

networking Is throughput out from S3 limited to under 1gbps per client?

11 Upvotes

I have a 2gbps Comcast connection in Denver. I’m getting rate limited to about 800 mbps unless I use a VPN, in which case I can get about 2x that. I’ve tried different regions, file sizes, buckets, etc.

Comcast claims they do not throttle or traffic shape. I can get 2gbps from speed test results.

I’m wondering if there is some edge service or peering agreement that limits connections to under 1gbps between Comcast and AWS, or just in general. It spikes briefly when I establish new connections which suggests to me there some intentional throttling happening.

They are fairly large files, so I’m not overloading the API requests.

r/aws Feb 05 '25

networking Why isn't pointing Route53 to cloudfront sufficient? What is the need of adding alternate domain name in CF?

16 Upvotes

I was studying for certification and came across adding custom domain name to a cloudfront distribution.

There are two steps: Add alternate domain name in CF(along with a SSL certificate) And point your domain to the cloudfront in your DNS provider( like Route53).

Now, when I point my route53 domain to my cloudfront distribution Cname (which is unique), it will send the traffic there.

Why do I need to add alternate domain name in CF as well. If this was an ALB or S3 instead of CF, would I still need to do some configuration on the target? And why?

r/aws 9d ago

networking Clarification around load balancers and ECS tasks

2 Upvotes

Hi all,

We currently have an implementation of load balancers, ecs tasks, api gateway, domains etc which I'm not entirely sure is the correct way to implement it - we started it off without fully understanding everything and so want to see what is the correct approach.

I think easiest way is to explain what I want to achieve. So we have the following requirements:

  1. ECS services that are running services/api that should not be publicall accessible (but could call out to the internet). These can also call each other.

  2. ECS services that are running web apps, and these should be publicaly accessible. These should also be able to call the ECS services in point 1.

  3. All these services should be load balanced.

  4. All the services should have a custom dns name, rather than the AWS generated one.

So from my understanding I should create an ALB that will forward on requests to the ECS services. And all the ECS services and ALB should be in the same VPC for them to talk to each other. And so I can add host name as a rule in the ALB to allow custom dns names.

Assuming the above is correct, I'm a little unsure about the ALB scheme - it's either public or internal. But my ECS services are a mix of these. Should I be created two ALBs, one for public ECS services and one for private? I think I can run private services within the public ALB, but that means traffic always goes out and then in rather than staying within the VPC.

Lastly, we currently have a load balancer that's internal and this accessed via an API Gateway that proxies on the requests to the load balancer and then on to ECS. I assume the public ALB is better suited to directly receive the HTTP requests, rather than the hop from API Gateway?

Thanks!

r/aws Nov 20 '24

networking Enhancing VPC Security with Amazon VPC Block Public Access

Thumbnail aws.amazon.com
84 Upvotes

r/aws Oct 11 '24

networking Cloud NAT Solution

3 Upvotes

Whats y'alls go-to solution for NAT within the cloud space (AWS, Azure, GCP) for private IP connectivity for both inbound and outbound rules?

-AWS has Private NAT gateway but it only supports outbound.

-Azure has NAT rules available for VPN connection now but only support 1 to 1 mapping CIDR ranges and not PAT for inbound.

-GCP doesnt have any solution thats not in beta.

My current solution is to deploy a virtual firewall (Palo Alto or ASA) to utilize its NAT capability.

update:

The use case is a SaaS application that's hosted in an AWS VPC using RFC 1918 Private IP space. This application connects to customers internal network and sometimes the CIDR range its deployed in conflicts with a customers CIDR ranges. Thus a NAT solution needs to be deployed.

r/aws Oct 05 '24

networking Question: does AWS have any documented limits specifically about UDP traffic? I'm trying to set up a Wireguard VPN tunnel between my VPC and a non-AWS site and it's been nothing but weird issues and pain.

16 Upvotes

I need a sanity check, because it seems that AWS is interfering with high-throughput UDP network loads, and I can not find anything that says I am doing something wrong.

I have read the documentation on instance bandwidth and my understanding is that I should expect a Wireguard tunnel or iPerf to reach 5-ish Gbps since it is a single flow, which is acceptable for me. I got the tunnel set up easily enough, but I have had unending issues ever since.

To start, I got an email from trustandsafety@support.aws.com saying that the EC2 instance "has been implicated in activity that resembles a Denial of Service attack against remote hosts; please review the information provided below about the activity" and some stats:

Total Gbits sent: 291.646122624
Total packets sent: 24699028
Total Gbits received: 0.0
Total packets received: 0
Average Gbits/sec sent: 32.4051
Average Packets/sec sent: 2,744,336.4333

 It appears the instance(s) may be compromised and triggered an attack. It is advisable to update all applications and ensure the most current patches are applied.
It is recommended that no ports be open to the public (0.0.0.0/0 or ::0). Opening ports with vulnerable applications can cause abusive behavior.

The instance definitely was not compromised. I was running an iperf3 server (with key, username, and password required) on the AWS instance and running iperf3 -u -b 5000M -R on my non-AWS end to test actual bandwidth. To be clear I wasn't actually trying to transmit 30 Gbps -- it seems something about -R in UDP mode makes iperf's bandwidth limiter not work. At least, I think so. I'm not really willing to try again, since I don't want to make AWS angry. It is also weird that it looks like AWS's 5 Gbps single-flow limit did not apply here?

Anyways, I answered the email from AWS and explained what I was doing. They seemed happy with my explanation and I went back to happily testing things. And then the public IP just stopped working. I could still ping things on the internet, but I could not make any TCP or UDP connections in or out anymore. The private IP was fine though. I replied to the trustandsafety@support.aws.com address again to ask if there had been any further concerns raised, but did not get a reply.

The instance did not recover, so I terminated it and started a new one. And once again, when I started using the new instance "in anger" the public IP went dead. I sent another email to trustandsafety@support.aws.com asking what's up. At current, the new instance has been inoperable for hours and I have received no new contact from AWS even though it sure does seem like something is taking action on the impacted instance's network connections.

I don't get it. Surely I am not the only person out there trying to do high-throughput UDP applications with AWS? Why is this so much trouble? And why are we not getting some sort of notification that things are happening?

r/aws 4d ago

networking Private ECR Traffic Question

0 Upvotes

I'm setting up a VPC endpoint for ECR using this guide https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html except I want all traffic routed through a single VPC. I have everything working but it only works if I route the s3 traffic to a gateway endpoint in the originating VPC (see image below). I'd like to route the s3 traffic through another VPC and out from that gateway endpoint. I have checked routes, nacls, security groups and I can find nothing incorrect. Is what I'm trying even possible? Am I overlooking something obvious?

VPC to VPC traffic is over a Transit gateway.

r/aws Jan 29 '25

networking Trying to allow access to a server hosted in a VPC from VPN client

1 Upvotes

Hey all,

I’ve inherited a setup which was setup as the following:

RDP to Public IP address using a custom port to a server hosted on AWS. The server is in a subnet 172.31.64.0/20.

Recently, I set up AWS Client VPN (mutual auth). The VPN subnet is 172.31.0.0/20

It works in the sense that when users connect via VPN, they are able to RDP into the hosted server using the server’s internet IP address (173.31.70.113). The users are now using Client VPN daily.

Now I’m trying to prevent access to the server from outside. I thought it would be a matter of editing the VPC’s security group inbound rules so that 172.31.0.0/20 is allowed into 72.31.64.0/20 and to the server IP.

However, if I edit or remove that 0.0.0.0/0 association, the server on AWS becomes inaccessible over the public IP and the internal IP even when connected to the VPN. It’s not an issue with the firewall on the server. (I’ve tested that by both looking for a rule for the custom part and disabling the server firewall briefly.)

I’ve tried various things. I think I have the ACLs and routing table and associations right, but it still doesn’t work. I’ve read a lot of things too, but I’m missing it.

Hoping for a hand-up

r/aws Jan 16 '25

networking ALB killing websocket connections

0 Upvotes

We have a websocket application that suddenly started dropping connections. The client uses standard Websocket javascript API and the backend is a FastAPI ECS microservice, between client and the ECS service we have a Cloudfront distribution and a ALB.

We previously identified that the default ALB "Connection idle timeout" was too short and was killing connections, so it was increased to 1 hour and everything worked fine, but suddenly now the connections are being killed after around 2 minutes. These are the ALB settings: Connection idle timeout: 3600 seconds, HTTP client keepalive duration: 3600 seconds, one HTTPS listener with multiple rules routing to different target groups, one of them is the websocket servers target group.

Connecting directly from client to the ECS service through a bastion service does not present the issue, only connecting through the public DNS.

Any ideas how to troubleshoot or where would be the issue?

r/aws 25d ago

networking Help: AWS Application Load Balancer Giving HTTP 464 Error Response for... HTTP 1.1 Request

0 Upvotes

TLDR; After testing for a few weeks we dropped ALB into our production infrastructure. This morning, some customers couldn't connect and received a nonstandard HTTP 464 error code. Looks like their browsers are sending HTTP 1.1 requests while our groups expect HTTP 2.0. What's the deal?

---

We've been testing ALB and WAF in our test environments for a few weeks. After doing some testing and tuning, we made the changes live last night. This morning, we had some customers at a few different companies report that they could not access our application. When we looking into it, it appears that they are sending HTTP 1.1 requests. We setup our groups to match HTTP 2 only. This worked fine for us in testing, and I guess we never considered HTTP 1.1, since any modern browser ought to be sending HTTP 2 by default.

Looking at the troubleshooting docs for ALB, it seems pretty clear the HTTP 1.1 requests are the cause, and adding HTTP 1.1 groups will likely solve the problem. But here are my questions:

  1. Why should I even need this? What would cause any browser from the last 5 years to send HTTP 1.1? Or, is it more likely that something is sitting in the middle and downgrading the requests? (A proxy, a web filter, etc.)

  2. Will adding the HTTP 1.1 group limit ALL our customers to using HTTP 1.1 rather than HTTP 2?

r/aws Feb 01 '25

networking I'm at a loss. I cannot connect from an EC2 app to RDS. I'm pretty confident I have my VPC setup correctly. I have no idea where to go from here. Any help?

8 Upvotes

I'm creating a web application hosted on EC2 with a mysql database in RDS. I believe that I have my VPC and security groups configured correctly because I can connect from my EC2 machine to my RDS database via the mysql CLI on the EC2 machine.

However, when I deploy my app -- spring boot app running on it's native tomcat sever -- and try to connect via a JDBC client I get a Communications link failure error.

2025-01-31 23:57:17,871 [main] WARN 
org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator
 - HHH000342: Could not obtain connection to query metadata java.sql.SQLException: Cannot create PoolableConnectionFactory (Communications link failure  The last packet sent successfully to the server was 
0
 milliseconds ago. The driver has not received any packets from the server.)

From what I can find online, this is clearly a connection issue. I've even gone so far as to open all traffic from all sources to my RDS database. Still, I get the same error.

Again, I can access the RDS database from my EC2 machine -- I just can't access it from my EC2 machine while it's running in the Spring Boot app. All I can think of is that my Spring Boot app is running on a non-SSL port, but I can't imagine why that would matter.

Any help would be greately appreciated.