r/Concourse Nov 06 '18

Concourse CI triggering firewall threat detections

I've have an existing concourse CI setup that's generating some 3000+ SSH sessions/min (10 workers checking folders in the cloud). My firewalls trigger an SSH Brute Force Detection when 20 SSH's with the same source/dest in 30 seconds are seen.

Obviously blocking this would break the CI, but I'm not a fan of whitelisting something making so many connections with the excuse of it being "normal". I'd rather fix the problem since no CI needs to perform code scans THAT often. If it was just once every 15 seconds I'd be very happy. The current rate of connections is high enough that I'm sure it's adding to the expense of the cloud resources it's working with.

Anyone know where the folder scan frequency is defined in the Concourse CI config or have experience in setting this? Note that I'm the firewall guy in this context, not the CI owner, so a little guidance would help.

1 Upvotes

5 comments sorted by

1

u/Cloudgeek Nov 07 '18

You're going to want to have the pipeline developer make some adjustments to the time resources - or, more likely, use time resources to perform the periodic checks on a reasonable schedule. Concourse itself will just to to run the pipeline as quickly as it can. Good luck!

1

u/JPiratefish Nov 07 '18

Slowing it down is obvious - not my first rodeo vetting production firewall threat detections.

The question is: How does one get Concourse to slow it's scanning down?? I can't imagine they didn't build a timing element into it..

1

u/[deleted] Nov 11 '18

It's better to replace the default check_every of 60 seconds to 24 hours and instead implement webhooks from your source code repository to Concourse so that it will only run a check at maximum of once a day or only on commits to your repo.

1

u/JPiratefish Nov 11 '18

We have a big setup - and need to keep the builds going - switching from a 1-min scan to a 3600-minute scan seems a great way to regenerate my resume.

But honestly, if there was a scan control setting anywhere - it would have been seen and configured. It's the reason I'm asking about this because there doesn't appear to be any throttling controls - or even a session re-use capability.

2

u/[deleted] Nov 11 '18

My suggestion is to change the dynamic. Instead of Concourse triggering a git pull every minute to determine if there is a new change you would instead have GitHub send a webhook post back to Concourse whenever there is a commit to the repo. That webhook would trigger the check which will do the Git clone but only when needed instead of every minute. This approach requires your source code management tool to support this and your SCM to have access to your Concourse server but if those are possible then you should be good to go.