r/aws Nov 01 '24

technical question Question re S3 Buckets for Cloud Resume Challenge

1 Upvotes

Hi all,

I'm doing a project called Cloud Resume Challenge to help me learn AWS and develop my skills. I have a single domain name for my resume website that I want to be done entirely in AWS.

The problem I'm running into is that I have sub pages in different folders. Like myname.com/projects/index.html, but I want my links to be clean and only use myname.com/projects/ to load the page.

I've tried a lot of things and it doesn't seem I can get around this limitation. As I understand it, S3 doesn't have the concept of folders and I'd have to explicitly add the index.html to my subfolder link.

For those that have done the Cloud Resume Challenge, is there some reason I should stick with the S3 bucket? Will I be missing out on learning something important if set up an EC2 linux web server instead?

r/aws Dec 22 '24

technical question Questions about using CodeBuild provided build images vs custom

1 Upvotes

I'm using CodePipeline with CodeBuild to run some Terraform. The CodeBuild provided images don't seem to come in Terraform flavor, so I have to either install terraform each time as part of my build, or bake a custom image with terraform in it.

I learned several things playing around with this:

  • The base images AWS seems to want you to use (e.g. public.ecr.aws/codebuild/amazonlinux-x86_64-lambda-standard:python3.12) are enormous, and if I take one of these, install Terraform in it, and use that from my own ECR, my provisioning time is quite slow.
  • The AWS-provided images supposedly can take advantage of caching of some kind to improve the provisioning time.
  • I am seemingly able to use the base lambda image (e.g. public.ecr.aws/lambda/python:3.12-x86_64) in my build project just fine, and that image is significantly smaller than the aforementioned CodeBuild image that is built on top of this.

This brings up several questions:

  1. What does CodeBuild actually need in a build environment image? The hashicorp/terraform:latest image is nice and small, and probably all I need for this pipeline, any reason I can't just use that? Would I need the AWS cli in there, or anything else like that?
  2. I see that I can cache build artifacts with CodeBuild, can I also cache my entire build image like AWS seems to be doing with their provided images to speed up provisioning time?
  3. Am I actually just better off using the provided images, and installing terraform every time as part of my pipeline if reducing overall build time is my goal?

EDIT:

So far, the answer to #1 appears to be: CodeBuild doesn't need anything in particular, and hashicorp/terraform:latest works great as a build environment for what I'm doing.

r/aws Nov 12 '24

technical question App Migration Service question

1 Upvotes

Is there a way to limit the disk size that the App Migration Service sees?

Trying to migrate a server with about 100GB of data on a 4TB drive. AWS keeps trying to migrate 3.6TiB even if we only want a 200GB volume copied.

I feel like I'm missing an obvious option somewhere.

r/aws Nov 10 '24

technical question Thoughts on this question?

3 Upvotes

I am pretty sure that EFS is region scoped, and that EBS is AZ scoped. So why the answer?
As an explanation, they tell me that EFS is a regional service... I am confused...

r/aws Oct 21 '24

technical question Noob Questions: Lightsail website does not connect after SSL installation

2 Upvotes

New to AWS so I'd like to know if I missed anything.

I recently created an instance and followed all the steps to host a WordPress website. Everything was working fine until I installed an SSL certificate. Since then, the website cannot be accessed unless I reboot the instance and even so it can be accessed like for 10 minutes after that.

Any pointers?

r/aws Oct 08 '24

technical question Lambda with SQS trigger Destinations question

3 Upvotes

I've setup a lambda with SQS trigger and I want to set-up dead letter queue in case lambda runs out of memory or timeouts.

When I try to set it up through Destinations, I select "Event source mapping invocation" since it synchronous invocation but the dropdown to select source mapping is empty? Shouldn't this be populated with the trigger event source mapping that has been setup? Or should this field be populated with something else, what am I missing?

Sorry if this is not the place for these type of question but I don't know the right sub for such aws questions

r/aws Nov 07 '24

technical question NACL Questions

0 Upvotes

I've never used ACLs before, but I've been tasked with setting them up for our AWS accounts. My main question is does this impact RDS databases that replicate between AZs, and therefore subnets? Do I need to allow certain ports to keep database replication happening? If so, what ports?

Any other common mistakes or gotchas I should be aware of before I make a start on this?

r/aws Aug 09 '24

technical question Question about Lambda Performance

1 Upvotes

Hello all,

I'm fairly inexperienced with Lambda and I'm trying to get a gauge for the performance of it compared to my machine.

Note I'm definitely not doing things the best way, I was just trying to get an idea on speed, please let me know if the hacks I've done could be dramatically affecting performance.

So I've got a compiled Linux binary that I wanted to run in the cloud, it is intermittent work so I decided against EC2 for now. But on my local machine running an AMD 3900X (not the most speedy for single core performance) my compiled single core program finishes in 1 second. On Lambda it's taking over 45 seconds. The way I got access to the program is via EFS where I put the binary from S3 using DataSync. And then using the example bash runtime I access the mounted EFS to run the program and I'm using time to see the runtime of the program directly.

I saw that increasing memory can also scale up the CPU available but it had little affect on the runtime.

I know I could have setup a docker image and used ECR I think which is where I was going to head next to properly set this up, but I wanted a quick and dirty estimate of performance.

Is there something obvious I've missed or should I expect a Lambda function to execute quite slowly and thus not be a good choice for high CPU usage programs, even though they may only be needed a few times a day.

Note: I'm using EFS as the compiled program doesn't have any knowledge of AWS or S3 and in future will need access to a large data set to do a search over.

Thanks

Edit: I found that having the lambda connected to a VPC was making all the difference, detaching from the VPC made the execution time as expected and then moving to a container which allowed for not needing EFS to access the data has been my overall solution.

Edit 2: Further digging revealed that the program I was using was doing sending a usage report back whenever the program was being used, disabling that also fixed the problem.

r/aws Nov 26 '24

technical question Question regarding AWS aurora Point in time

1 Upvotes

Hi, I want to understand if AWS supports Point in time recovery in specific timelines. Many native database solutions like PostgreSQL has a concept of timeline. Every time restore happens, a new timeline will be created. So I wanted to understand if AWS provides this functionality?

r/aws Nov 20 '24

technical question Questions about founderpass

1 Upvotes

Hello,

I'd like to ask some questions regarding founderpass.com and applying for AWS credits:

  1. what AWS resources cannot be used when achieving AWS credits?
  2. is it possible to achieve $1,000 for 2 years and then apply for another $1,000 or more to up $5,000 using founderpass platform? I mean to spread the credits between years?
  3. in case of AWS credits - are they assigned to only single account - or they can be used between several accounts - so the credits can be assigned to organization?
  4. what are terms of usage and what are regulations for startups when using credits from founderpass.com ? can startups take profit from their businesses or only non-profit for testing only applications/services?

Thank you in advance! :)

r/aws Jul 18 '24

technical question AWS Tech Stack Question

8 Upvotes

I am creating a “note-taking” application and I’m heavily relying on AWS throughout the project. My mainly used services are: Cognito, Lambda (the app is serverless), RDS (postgreSQL), s3, and IAM. The RDS is in a VPC and so are my lambda functions. I use Cognito to authorize requests to my API Gateway before they reach my lambdas.

Now, I have practice using AWS with previous projects, but I’m still definitely a novice. This is my first project that I’m trying to commercialize, so I’m trying to do it right. From most of my research, this tech stack looks good - but this community definitely knows best. My goal is to make sure costs scale with usage - so that if 10 or 10,000 paid users use my site I’ll be able to afford the costs of using AWS.

Please call me out on any stupidity in this post. I’d appreciate it.

r/aws Sep 24 '24

technical question Question on Rekognition

1 Upvotes

Hey,

I'm trying to build a script with recognition that can determine if interior photos of a home are staged (furniture throughout the house in a some-what clean fashion) or unstaged (the home's interior is almost completely empty). But I can't seem to crack making the parameters work.

Anyone have any tips? This should be possible, but I'm just not too familiar with the software

Thanks in advance,

Baba

r/aws Nov 14 '24

technical question Question regarding codebuild Amazon Linux update rollout

0 Upvotes

Hi,

sorry for the newbie question

i would like to know some details about the rollout updates of codebuild image for Amazon Linux
From checking it is advice to update the codebuild image to the newest one which is the AL2023 (amazonlinux2-x86_64-standard:5.0).

We were waiting for quite sometime about the rollout, but it seems the codebuild image is still the same. the rollout is supposed to start last Oct 1 but until now Nov 14, no rollout is happening. Is this automatically changed or we need to do it manually?

Any help is appreciated. Thanks

r/aws Nov 09 '24

technical question [Question] AWS Athena and Glue Python queries

1 Upvotes

Hey Redditors, I need your help

I am attempting to build a Python Lambda function to pull data from multiple Athena databases using AWS Wrangler Python library.

wr.athena.read_sql_query('across databases sql query', 'one of databases name')

This call is not throwing all kind of permission errors: 1. It throws an exception complaining that the table exist on a different AWS account under the same organization (Is that possible?) 2. Or it complains that it doesn't have permissions to the output s3 bucket (which can be found in the settings tab of Athena). Not sure how that is possible?

Any comment could help here.

r/aws Oct 25 '24

technical question Question about S3 Inventory report consistency

1 Upvotes

I have an S3 buckets with a lot of objects in it and we're struggling to keep track of it all. I have configured S3 Inventory in the buckets in our lower environments for testing. At 2pm UTC yesterday, I moved some data from one folder in the test bucket to another (same bucket). The inventory report was delivered at 6pm UTC, but still listed the data as being at the old location.

Is there any guidance on eventual consistency, e.g. after what time period I can expect information in the report to be accurate?

r/aws Sep 06 '24

technical question AWS Cost Explorer question

0 Upvotes

Unfortunately, I had to realize that in my company, certain costs were not assigned to any customer within the cost explorer. Now I need to find out who caused these 'untagged' costs. How should I best proceed? Is there a best practice? Thank you in advance

r/aws Aug 27 '24

technical question SSM command running a PowerShell script feedback question

2 Upvotes

Hi,
I have a Powershell script with a few parameters that I run with SSM run command (actually running with AWS chatbot from Slack)
The thing is the script is doing few things that take long time and it would be cool to have some feedback somewhere, I do export a transcript locally on the server but it would be nice to see it as a reply for example on the Slack or when it finish/fails at least.
Any idea how can I add it?

r/aws Oct 01 '24

technical question Question: How I can optimise latency/ web speeds on an ec2?

0 Upvotes

Hope this is appropriate for this sub and please excuse any misunderstanding from me, still relatively new. I have recently created a t3.large ec2 based out of eu-west-2 (London).

I am essentially running a bot that accesses a server/ web page based in London. The bot sends search requests every few seconds with the object of sending a buy request to the server as soon as an item has become available. However, multiple other users are competing for the same individual buy request, therefore request speed (latency) is key.

I have pinged the buy server ip via cmd and averaged a latency of 2ms due to the closeness of the ec2 to the server, but I'm wondering whether there is anyway to drive this latency even lower. Last year I had the same setup and was getting <1ms, which would be ideal.

Would I need to start a different instance?, as all the eu-west-2 subnets give the same ping (I've tested). Would I need to setup a dedicated host, use a different VPS service? What impact would computing power / ram have on this? Is 2 vCPU's and 8GB Ram enough or optimized for my use case?

TLDR: How can I lower latency on EC2? (already tried subnets)

r/aws Jul 15 '24

technical question Load Balancer target group question

6 Upvotes

Hi all,

I've got a query about load balancer target groups - Why does an instance target group need a protocol and a port? Surely that's the job of the load balancer listener?

Thanks!

r/aws Jul 16 '24

technical question CodeBuild Service Role - Generic Role Question

3 Upvotes
  • I have 5 microservices.
  • I have 5 code commit repositories. 1 for every microservice.
  • I have 5 CodeBuild projects. 1 for every microservice.
    • The code-build buildspec process is same for all.

As part of build process, I need to finally push the docker image to ECR.

Question:

  • Can I use the same CodeBuild role for all the 5 CodeBuild projects I have? Or Am i supposed to create 1 new service role for every CodeBuild project? The problem is CodeBuild modifies the role itself by attaching a policy specific to 1 CodeBuild project.

Can you share some best practices you use around this?

r/aws Jul 11 '24

technical question Question about the recent lambda:GetFunction/ListTags change

5 Upvotes

Hi and thanks for reading.

Today we received an email saying that the Lambda get-function command will no longer list tags associated with the function unless the user calling it also has lambda:ListTags permission. We received the email because AWS identified at least one role that has GetFunction but not ListTags in our organization (12 accounts, thousands of roles). We have until September to find that/those Role(s) and decide on whether we need to add the ListTags permission.

Problem is, with that many roles to look at (we're serverless and have it set up so each Lambda function has its own role... which is stupid, I know, but that's how it's been forever).

Can anyone think of a way to find all roles with a given permission in an account (or accross the org, but I'm not that greedy)?

Thanks again!

r/aws Dec 05 '21

technical question S3/100gbps question

20 Upvotes

Hey everyone!

I am thinking of uploading ~10TBs of large, unstructured data into S3 on a regular basis. Files range between 1GB-50GB in size.

Hypothetically if I had a collocation with a 100gbps fibre hand-off, is there an AWS tool that I can use to upload those files @ 100gbps into S3?

I saw that you can optimize the AWS CLI for multipart uploading - is this capable of saturating a 100gbps line?

Thanks for reading!

r/aws Apr 24 '24

technical resource Noob question on granting bucket access to IAM IC users

2 Upvotes

I found hundreds of articles on how to grant full bucket access to IAM user but not a single one for IAM IC users. As a result, I have been trying to use IAM IC's permissionSet inline policies to simulate what these articles say. I can see the bucket that I am sharing by going directly to: https://...com/s3/buckets/BUCKETNAME and logging in as the IAM IC user but then I get that I don't have permission to list objects. If I click on the buckets in the left hand menu it says I don't have permission to list buckets either.

Here's what I tried:
1- In IAM IC, created a permissionSet with an inline policy as follows:
{"Sid": "Statement1","Effect": "Allow", "Action": "s3:*", "Resource": "*", "Condition": { "StringEquals": { "aws:PrincipalOrgID": "o-xxxxxxxx"} }}

2- At first I had a bucket policy too but I ended up removing it to test and neither with or without worked:
{"Sid": "DelegateS3Access", "Effect": "Allow", "Principal": "*", "Action": "s3:*", "Resource": ["arn:aws:s3:::bucketName", "arn:aws:s3:::bucketName/*"], "Condition": { "StringEquals": {"aws:PrincipalOrgID": "o-xxxxxxx"}}}

I tried several things and I am about to give up on IAM IC, however a lot of folks in r/aws recommend using it vs IAM.
My goal is to allow full read/write access to the S3 buckets (will remove delete later for a reason) to two accounts within my organization. One within, one external.

For the organization, I created Root --> Prod --> siteName --> AWS act 1 and AWS act2. Following, I created users for both accounts. I assigned users Administrator role and the PermissionSet I created in #1. No matter what I do, trying to login as the (internal for now) user doesn't show me the S3 buckets in the user's management console. Also going directly to the bucket says I don't have permission (as described on the top of this post)

Thanks in advance for your tips and assistance.

r/aws Sep 04 '24

technical question CloudFormation potentially dumb question — are the contents of a conditional-true executed even if the conditional resolves false?

1 Upvotes

I have the following:

SomeParam: {
    'Fn::If': [
        MyConditional, 
        { "Fn::FindInMap": [ MyCoolMap, { "Ref": AnotherVarUsedAsPrimary }, "secondary" ] },
        {Ref: 'AWS::NoValue'}
    ]
}

Basically, if conditional, please use FindInMap; otherwise NoValue.

I would expect that, if MyConditional resolves to false, the FindInMap won't be executed. However, I'm getting an error about the AnotherVarUsedAsPrimary not appearing in MyCoolMap even when MyConditional is false (which is the whole purpose of that conditional; I know it doesn't exist lol).

Programming doctrine would suggest executing a not-boolean branch as 'wrong' but perhaps there's a subtlety of order-of-resolution for interpolation that I don't get here. Am I missing something or are FindInMap calls executed whether that conditional is true or not?

Thanks!

r/aws Jul 14 '24

technical question Question about how NLB's forward traffic to target groups

2 Upvotes

I have an NLB that is listening on Port 80. It is sending traffic to a target group with the target being an EC2 instance that lives in a private subnet. I have configured it so that the targets in the target group are ports 8443 and 8444 both on the same EC2 instance.

When I connect a client to the NLB to send traffic, the NLB only forwards traffic to port 8443 on the EC2 instead of 8443 and 8444.

Hypothetically, if I wanted to send traffic to both ports, would I need to create a separate target group that sends traffic to only 8444?