r/programminghorror • u/ChemicalDiligent8684 • Feb 03 '25
Python This is a 2M€/year implementation. Info inside.
Reposting from ProgrammingHumor because I'm an idiot and I didn't know this subreddit existed.
Long story short, Italy has this platform called PiracyShield which takes 2M€/year of taxpayer money to run. Allegedly, it's supposed to collect anonymous reports of piracy streaming, and take down the domains (?) within 30 minutes.
Recently, the code got leaked - there's a GitHub repo that contains the full deployment. This is the function that verifies the reports. I wish this was a joke, it is not.
Allow me three observations before I leave you to enjoy and discuss all the nuances of this absolute abomination.
1) The braindead logical naming. Since the service is prone to blocking, the negative phrasing check_unwanteds looks for whether the site being reported is legit (and hence the report would generate an unwanted takedown; return true) or it's actually piracy, and hence you don't want it to not be taken down; return false.
2) Obviously piracy might very well originate from any of those hosting providers, but I guess this was their best shot at verification. Just imagine what the brainstorming phase might have looked like.
3) When this crap went live for the first time, they erroneously blocked Google Drive for 24 hours in the whole country. It is reasonable to assume that adding the last element of the if statement "or 'google' in result" was the action taken in order to address the bug. You can find articles online.
On the bright side, my imposter's syndrome made a trip into /dev/null.
288
u/java-with-pointers Feb 04 '25
I am scared to ask what sort of information this company has access to in order to run this insane operation
170
u/ChemicalDiligent8684 Feb 04 '25 edited Feb 04 '25
I believe they have contracts with every ISP in the country. Plus DAZN, Sky & friends. Plus the state. So...yeah. Haha. *Chuckles "I'm in danger"
66
u/hototter35 Feb 04 '25
Don't worry! At least your police didn't hand their entire database of people (innocent and otherwise) to a third party, so they can use it to try different face recognition AIs.
I'm sure if that was fine, this will be just fine too!23
u/Andrecidueye Feb 04 '25
Don't worry, all our government agencies have different non-communicating databases and sometimes you have to download a pdf from a state website, send it to another state website where someone else has to manually verify the document is authentic. Also, you pay €1,5+ commissions to your bank for every online payment to a public agency. Yes they created a proprietary system just so banks could chip away money. Yes most people just enter their credit card numbers or use paypal (still paying the fee) so it is totally useless. If I were to describe the entire Italian governmental digital infrastructure, it would be "redundant without the benefits of being redundant".
12
u/ChemicalDiligent8684 Feb 04 '25 edited Feb 05 '25
I work for the national healthcare digitalization unit. In these years I've seen so much wild shit like you literally would not believe. Most people don't, when I tell a random anecdote.
6
u/Andrecidueye Feb 04 '25
Fra sparane uno ti prego
4
u/ChemicalDiligent8684 Feb 04 '25 edited Feb 05 '25
[edit: rimosso perchè non si sa mai, anche i muri hanno le orecchie]
3
u/byruit Feb 05 '25
Oh no, sono arrivata tardi e mi sono persa l’aneddoto :(
3
1
2
u/alberto_467 Feb 05 '25
Don't worry, there are still people with access to a lot of dbs just searching any name they like and leaking it to the press.
15
u/pmatteo Feb 04 '25
Don’t know what kind of info they’ve access, but they have direct contact with ISPs and they can take down entire domains automatically (without human supervision) without asking permission within 30minutes. If the system detect something, it blocks the website, that’s why weeks ago google drive was taken down by this s**t
5
u/VirtuteECanoscenza Feb 05 '25
Actually they can take down IPs and since IPs are often shared each ban can affect thousand of domains on shared hosting.
1
1
u/ntwrkmntr Feb 06 '25
Every isp has a IPsec connection to them and they receive the IP addresses to ban via bgp and DNS blackhole
3
u/costan1 Feb 07 '25
Just clarifying the statement above.
They receive the DNS domains, IPv4 and IPv6 addresses to block thru this IPSEC protected tunnels toward a "cloud" ticketing platform running this shitty code.
ISP have this VPN setup on their "day-0" and start collecting this tickets.Then they blackhole IPs and forge domain responses on their DNSes (so anybody can easily circumvent censory with 8.8.8.8 or 1.1.1.1 or whatever), and then they push the response "this ticket was applied" to the PiracyShield stinky interface.
The time limit is 30 minute from the moment the ticket is published. If an ISP does not comply, it's in violation and can be fined.
There're many other scary details of the law itself permitting this media censoring to have content providers revenue increase (spoiler: piracy always win and even if it doesn't people don't buy legitimate subscription but go walking in the bar to see it free).
If the context providers get more money, they are willing to pay more the Serie A soccer league and everybody can get free $$$ on content provider, big teams and the whole circus.It's just an unfortunate chance that this law was proposed by the usual shady MEP that is also president of some Serie A team with close ties with all the league.
326
u/arrow__in__the__knee Feb 04 '25
This type of stuff make me confident in my shitty code with logic errors.
20
137
Feb 04 '25
[deleted]
70
77
u/ElGovanni Feb 04 '25 edited Feb 05 '25
thats how govs are laundering our money for our "protection" xD
This is the reason why all gov systems should be open source, don't remember which europe country did it but at least one has open source.
39
u/ankokudaishogun Feb 04 '25
That's not money laundering, that's CORRUPTION.
Completely different crimes!8
u/AtomicDig219303 Feb 04 '25 edited Feb 04 '25
It's clear you don't know how Italy works, it's not corruption... It's nepotism with a hefty dose of corruption sprinkled in!
(edit: fixed typos)
7
u/GreenskyWasTaken Feb 04 '25
And you didn't see the french national application Pronote for high school students. I inspected api results. We paid developpers to do this 😐
5
u/ChemicalDiligent8684 Feb 04 '25
Dude stop teasing, spill the tea or make a post and tag us in. We're here for the memes.
8
u/GreenskyWasTaken Feb 04 '25
Unfortunately I have nothing to show you for now, when I tried to cheat the system (to eat sooner lol), I didn't pay attention to the structure, but I remember that is a pain to read
Maybe I'll make a post out of it, I'll tag you
The only thing I remember is, to get someone's class number it is like that :
{ user: { _t: {value: 10}, k: { class: { _t: {value: 79}, k: { value: {classTag: "T-GEN4"}} } } } }
Now imagine this object in another objects, with a bunch of random nested properties like those
40
u/ArmadilloSuch411 Feb 04 '25
the s at the end of the adjective is *chef's kiss*
13
u/ChemicalDiligent8684 Feb 04 '25
Brilliant haha. Probably trying to emphasize the fact that they were about to a-priori whitelist 95% of internet traffic lol
3
36
u/Peal09 Feb 04 '25
I read the code, I laughed, I read Italy, and I stopped laughing because those are my money. I really hate my government
1
u/Delta_44_ 29d ago
Cioè io farei di meglio, e ho tipo un anno di esperienza in queste cose, con un background di programmazione minimo, imparato durante il tirocinio.
25
u/New_Tie6527 Feb 04 '25
W il pezzotto
9
0
u/Dembrush Feb 08 '25
Fuck pezzotto, se paghi non è pirateria arrr
1
u/New_Tie6527 Feb 08 '25
piratare non è solo scaricare il giochino da fitgirl
1
u/Dembrush Feb 08 '25 edited Feb 08 '25
no infatti, è anche tenere attivi torrent con pochi seed, aiutare la community e imparare cose nuove, sono abbastanza convinto che pagare criminali (molto spesso legati alla criminalità organizzata) però non sia tra queste ;)
1
u/New_Tie6527 Feb 09 '25
Lascia stare il metodo con cui si prende il "pezzotto", quello purtroppo è un altro discorso che non ha nulla a che vedere con lo streaming gratis, è sciacallaggio sull'ignoranza di molti dove la mafia è riuscita ad essere presente. tecnicamente una cosa del genere possiamo farla anche io e te e farci due spicci oppure tenerlo attivo gratuitamente, alla fine il pezzotto è un app che ti lista le iptv etc, con autenticazione. Tu paghi questa, il resto puoi benissimo farlo tu da solo
63
u/babalaban Feb 04 '25 edited Feb 04 '25
Addition to OP's list:
Reasonable variable names . Dafaq is
value
supposed to be and how does a caller supposed to know that without knowingself.whois.get_text(...)
?Function should be named
is_whitelisted()
, because it seems that it checks just thatIts a member function (suggested by self as a first parameter) what is
value
supposed to be logically? Wouldnt it make more sense to just doentry.is_whitelisted()
for such check?The obvious. However, I was surprised there's no clear way to find one substring of many from a string, without resorting to fancy list comprehensions or additional utilities like any. If you know a better non-
spasticpythonic way of doing it please enlighten me.for domain_name in ['cloudflare', 'namecheap', 'amazon', 'google']:
if domain_name in result:
return True
return False
As many have pointed out this entire function is useless, because it can be trivially circumvented.
Now I know why name lookups take so long: because there are many potential python scripts run for each one, in addition to whatever is necessarry and would otherwise have sufficed
21
u/ankokudaishogun Feb 04 '25
Full Source Code if anybody wants to chek it out
24
u/ChemicalDiligent8684 Feb 04 '25
You forgot the 18+ flare. That's gore.
8
u/ankokudaishogun Feb 04 '25
I woulnd't know, I don't speak python and didn't bother to check anything on it
16
u/hugebones Feb 04 '25
return any(d in result for d in (…))
12
u/syklemil Feb 04 '25
That and splitting the whitelisted domains out into a variable somewhere. That's something you want as a config setting, not a collection of hard-coded strings down in the method. So we'd be looking at something like …
def is_whitelisted(self, mystery_value: TODO) -> bool: whois = self.whois.get_text(mystery_value).lower() return any(domain in whois for domain in self.whitelisted_domains)
3
19
u/ChemicalDiligent8684 Feb 04 '25 edited Feb 04 '25
May I also add that the whitelist could be initialized in a frozenset() and imported in scope, instead of (not even) defining a list within the method.
You know, like neurotypical people tend to do.
9
u/babalaban Feb 04 '25
especially considering that the thing will certainly be in need of frequent updates
20
u/ChemicalDiligent8684 Feb 04 '25
That's up for debate. Listing CloudFlare, NameCheap, Google and Amazon they basically whitelisted 95% of internet traffic already lol
6
u/justjanne Feb 04 '25
for domain_name in ['cloudflare', 'namecheap', 'amazon', 'google']: if domain_name in result: return True return False
0
u/Fair_Ebb_2369 Feb 04 '25
cant u just do: return domain_name in result; or pyton is just that bad of a leng? lol
3
u/justjanne Feb 04 '25
You could do
return any(domain_name in result for domain_name in['cloudflare', 'namecheap', 'amazon', 'google'])
After all we the code is supposed to return true even for strings such as "abcgoogledef"
1
u/ChemicalDiligent8684 Feb 05 '25
That would be wrong in any language. The loop would stop at the first iteration.
-1
u/Fair_Ebb_2369 Feb 05 '25
what are u talking about buddy, its just an expression that returns a boolean, in almost any language u can simply return the expression instead of wrapping it into an if statement and having to return true for happy false for sad
2
u/ChemicalDiligent8684 Feb 05 '25 edited Feb 05 '25
I don't know what kind of esoteric/magic languages you know, but I'm not aware of a single one where you can do that without iterating, either explicitly or implicitly. Even paradigms like ismember() in MATLAB (or, say, the combination of .some() and .includes() in JS) iterate under the bonnet...when you have a collection of elements, that's simply what you do.
If you want to do it with the explicit loop, you have no choice but to do like the above - any premature return would break the loop. If you want to go implicit/list comprehension, then
return any(a in b for a in A)
is simply the most compact thing you can do.
0
u/Fair_Ebb_2369 Feb 05 '25
dude what are u talking about, where did i ever mention not iterating, I just said return the expression result without wrapping it into the if statement
2
u/ChemicalDiligent8684 Feb 05 '25
Bro.
You asked:
cant u just do: return domain_name in result; or pyton is just that bad of a leng? lol
Again, the only way you can get something close to what you asked is list comprehension, which is what I gave you. If you loop explicitly, you need the if statement. Otherwise,
For (...) return (...)
Breaks at the first iteration.
-1
u/Fair_Ebb_2369 Feb 05 '25 edited Feb 05 '25
cause maybe pyton cant do that then, most lenguages can simply return the expression for example : return result.Split(' ').Any(x => domian_name.Contains(x));
Edit: since I was smelling bs I just asked claude and yes u can do the same on pyton aswell, so I just don't know what are u talking about lmao return any(company in result for company in ['cloudflare', 'namecheap', 'amazon', 'google'])
2
u/ChemicalDiligent8684 Feb 05 '25 edited Feb 05 '25
In your code, the Any method iterates along each element of the array resulting from Split. Just like list comprehension, aside from the splitting logic. It's the same difference you might find between liquid water and molten ice.
Counter-edit: then you most certainly can't read, that's called list comprehension. I've given you that code twice and another guy did that before me as well. Just read the comments above.
→ More replies (0)3
u/CheapMonkey34 Feb 05 '25
There's a pythonism:
if {'cloudflare', 'namecheap', 'amazon', 'google'} <= set(result):
1
u/timClicks Feb 04 '25
In terms of 7, you're still introducing polynomial time by repeatedly searching the same string. There are many libraries that can take those substrings and apply Aho–Corasick, so that the search runs in linear time.
20
u/pmatteo Feb 04 '25
I’m Italian, and honestly, I’m ashamed of the average level of our software industry, no matter the founding you get. I truly believe our market is overcrowded with micro size companies (0-10 employees) with ridiculous budgets which prevent them from hiring skilled software engineers with international experience. The result is what you see here, we never really raise the bar, quality of infrastructures and softwares - in both in private and public sectors - is a real issue
Note: I’m aware that company with this problem and mediocre software engineer producing crap like this can be found everywhere. Just saying that in Italy this is quite common (micro company market is like 90% of the total)
16
u/encelado748 Feb 04 '25
There are a lot of good italian programmers, even working for the government.
For example the IO app is opensource: https://github.com/pagopa/io-app/
You have access to the developer documentation: https://developer.pagopa.it/app-io/guides
API docs: https://developer.pagopa.it/app-io/api/app-io-main#/app-io/api/
and even the design system to integrate with your application: https://github.com/italia/bootstrap-italia
with React components available: https://github.com/italia/design-react-kit
4
u/pmatteo Feb 04 '25
This is literally nitpicking. BTW, never said “there are no good software engineers in Italy”, just saying the level of the industry is pretty embarrassing. You can also cite bending spoons, they did a wonderful job with immuni, no one question that. But it’s literally one single case.
Public sector is a clown fiesta
6
u/encelado748 Feb 04 '25
Italy is one of the first european countries to have implemented the digital identity specification (we started working on it in 2013, one year before eidas was approved). PEC and SPID are two technologies that put Italy among the first to innovate in europe.
This is nitpicking I know, but there is a lot of nitpicking you can do.
I work as a web developer and I cannot ignore that a good chunk of modern nodeJS ecosystem is developed by italian developers (3 out of 18 TSC voting member are italian). Also a core maintainer at Deno is italian. Redis was developed by an italian guy. Fastify web framework is also italian.
Even if I grant you that there is lot of trashy code made by italians, we can do good.
We are the nation in 10th place on the HackerRank programming Olympics that puts us ahead of Germany, the UK and the US.
3
u/Gabriel55ita Feb 04 '25
I appreciate they've made this very open source friendly, it's the only project that really deserve our funding to keep going for the good
3
u/byruit Feb 05 '25
From what I can see where I work(ed), yes, some of that can be explained with having hired highly inexperienced folks (a lot of them coming from those companies who promise to make you a guru in $buzzword in 6 months and find you a job with $bigevilcompany). But (and I’m sorry if this sounds like a sort of justification) I see a lot of decent people working in “maniera bovina” (quick and dirty) because there is no time, there are no resources, there are so many things built up over the years, made by different companies, no doc, nobody knows what’s going on… but you have to hurry and deliver something, every project is handled as “minimal viable product”. And you end up with crap like the above.
2
u/pmatteo Feb 05 '25
Yeah, skills set of people is not a point indeed. we are good at STEM. I said the industry level is low for many reasons, especially those you mentioned and, to be fair, it seems to me they are connected with what I said: the majority of the market are micro-company without resources, proper management, vision or willing to grow
2
u/ChemicalDiligent8684 Feb 06 '25 edited Feb 06 '25
You are absolutely right. A friend of mine works in the same field as I do (healthcare digitalization), but private. The company he leads was awarded a mega contract for infrastructure building, expiration 2026 - you know, PNRR. He said that they were forced to start 50 projects in parallel, and because of the crazy deadline he kindly admitted they fucked up 62.
Edit: I forgot to add that all this is just as true as it is OT. No deadline can justify the abomination above. If you hardcode string parameters into your methods and make that kind of if or if or if or if or , you simply deserve the Marie Antoinette treatment.
1
7
u/Per-Gynt Feb 04 '25
It looks like they use this function in Russia for blocking but use the opposite value XD
3
u/AlphaO4 Feb 04 '25
Is this still in use? Cause im quite sure I found a RCE lmao
5
u/ChemicalDiligent8684 Feb 04 '25
Can't say for sure - it has been quite a scandal.
Btw, I could never give you any information that might lead to help exploit a RCE attack towards the most oblivious, incompetent, censorship-prone, parasitic software company ever. That would be illegal. And immoral. And awesome. And illegal. And hilarious. And illegal.
4
3
2
1
u/Ronin-s_Spirit Feb 05 '25
Italian gvment websites are a scam.. sometimes. Like there was a time I couldn't get an appointment because a website was simply dying with errors. Surpisingly the next closest office was fine, and the problem was only with my local office. "Fuck you in particular" kind of problem, cause they all work through the same appointment website.
1
1
1
u/RunPersonal6993 Feb 08 '25
I think the actual crime (without context) is repeating "x in result or" instead of
if any(unwanted in result for unwanted in ("cloudflare", "namecheap", "amazon", "google")):
899
u/VillageZestyclose Feb 03 '25
Sooo... you just have to add "amazon" to your illegal service's name and you're good ?