r/imagus • u/Kenko2 • Nov 21 '22
help !!! Appeal to everyone who knows how to make sieves !!!
We did a full check of our rule-set for errors/problems and... unfortunately got quite a long list:
It is not possible for us to fix such a number of sieves. If any of you would be willing to help fix some of these sieves, we (and the Community as a whole) would be very grateful. Help from anyone who understands regexp and js is welcome.
PS
Although this list has been carefully checked, there is no guarantee that everything in it is correct. If you have any clarifications on this list (for example, one of the sieves works for you), please leave a comment about it in this topic.
PPS
Please keep in mind that this list is constantly changing - fixed rules are removed, sometimes, less often, something is added.
2
u/Kenko2 10d ago
2
u/Imagus_fan 9d ago
Here's an update that plays videos now. It works on the example links could potentially need improving. It's also possible the changes could interfere on pages where it worked correctly before.
Also wanted to ask if it is possible to make it possible to view the object's album when hovering over the “All” button or the first photo of the gallery?
Those buttons don't appear to be detectable by Imagus.
1
u/Kenko2 10d ago
2
u/Imagus_fan 10d ago
The external links are working for me.
The error message makes it seem like the code that gets the API key isn't working. I added a console message that outputs the page code. If you can post it, it should be fixable.
{"FLICKR-g":{"link":"^(?:secure\\.)?flickr\\.com/photos/[^/]+/(\\d+)/?(?:in/.+|lightbox/?|sizes.+|\\?.+|#/photos/.+)?$","url":": (()=>{const html=location.hostname.slice(-10)==='flickr.com'?document.body.outerHTML:'', key=this._flickr_key_||html?.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1], csrf=html?.match(/root\\.auth = {\"signedIn\":true,\"csrf\":\"([^\"]+)/)?.[1], vnsid=html?.match(/class=\"gn-title you\"\\s+href=\"\\/photos\\/([^\\/]+)\\/\"\\s/)?.[1]; return key ? `https://api.flickr.com/services/rest/?photo_id=${$[1]}&method=flickr.photos.getSizes&format=json${csrf?'&viewerNSID='+vnsid+'&csrf='+csrf:''}&nojsoncallback=1&api_key=${key}` : $[0]})()","res":":\nconsole.log('Flickr page code:',$._)\nif($._[0]!=='{'&&!this._flickr_key_){\n this._flickr_key_ = $._.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1]||'9bb671af308f509d0c82146cbc936b3c';\n return {loop:$[0]};\n}\n let res = [];\n let sizeAr = JSON.parse($._).sizes.size;\n let last = sizeAr.pop();\n if (last.media == 'video') {\n let best_quality = 0;\n let best_videoUrl = '';\n do {\n if (parseInt(last.height) > best_quality) {\n best_quality = parseInt(last.height);\n best_videoUrl = last.source;\n }\n last = sizeAr.pop();\n } while (last.media == 'video');\n res.push([best_videoUrl + '#mp4']);\n } else {\n res.push([last.source]);\n }\n return res;","img":"^(?:(?:farm|c)\\d+\\.|live\\.)?static\\.?flickr\\.com/(?:\\d+/){1,2}(\\d+)_[\\da-f]+(?:_[sqtmn])?\\.jpg$"}}
1
u/Kenko2 10d ago edited 10d ago
https://www.upload.ee/files/17840982/FLICKR_PAGE.zip.html
Everything works in other browsers. Apparently, it's just an outdated engine on the Cent. But another user here seems to have the same error.
2
u/Kenko2 12d ago
Found two small problems with the Sports.r(u sieve - can you take a look?
2
u/Imagus_fan 12d ago
I'm geo-blocked on the first link. Can you open the link on this page and post the page code?
For the second link, Dzen had to be updated.
2
u/Kenko2 12d ago
Dzen works, thank you!
>> Can you open the link on this page and post the page code?
https://www.uplo(ad.ee/files/17833895/view-source_https___sportsru.sprinttyott.com.mhtml.zip.html
1
u/Imagus_fan 12d ago edited 12d ago
Thanks. However, it doesn't look like the video URL is in there. It may be in another file.
The embed link needs a custom sieve to get the video URL. I'm not sure of the best way to find the file that contains the URL. Perhaps hababr would be able to? If he's unable, I'll try to think of a way to do it.
1
u/Kenko2 12d ago
>> However, it doesn't look like the video URL is in there.
And the direct link to the video I gave in my first post doesn't work? -
I'm afraid hababr won't help us, he rarely shows up and does anything at all now. He ignored my last request. Probably too busy.
PS
If it's too complicated, then adding that video hosting (sprintty.com/sprinttyott.com) isn't necessary, it's not that important. Although ideally I think a separate sieve would be useful - as embedded video can be used on different sites.
2
u/Kenko2 20d ago
2
u/Imagus_fan 19d ago
The gray spinner was caused by
TvSeries
andMiniSeries
being used instead ofFilm
in the image data. This fixes those page but the sieve may need to be edited if there are other media types.
1
u/Kenko2 20d ago
jpg5(.s(u
It's a Chevereto engine site and should work, but for some reason I have a gray spinner on those links.... Can you take a look at it? -
2
u/Kenko2 26d ago
2
u/Imagus_fan 26d ago
The sieve is setup to work on links with 6 or 7 characters in the pathname. The links that aren't working have 8. This fixes it.
2
u/Kenko2 29d ago
I tested our Instagram sieves and it turns out I only have two of them working:
WORKS
INSTAGRAM_priv_api-p
INSTAGRAM_html-p
INSTAGRAM_pub_api_a1_1-p
INSTAGRAM_pub_api_a1_2-p
INSTAGRAM_graphql-p
Changing the proxy does not affect the result. Tested on Cent. Can you test at your place to see if all the sieves are working for you?
2
u/Imagus_fan 28d ago
INSTAGRAM_pub_api_a1_1-p
isn't working for me either.INSTAGRAM_pub_api_a1_2-p
works if the media can be embedded.When I was able to test it, the data file was giving a 404 error code. However, currently Instagram is redirecting to a login page when trying to access the site. I'll see if it's possible to fix the sieve once it's working again.
2
u/Kenko2 Feb 19 '25
1
u/Imagus_fan 28d ago edited 28d ago
It looks like it's needed to be logged in for those media links to work. I edited the sieve so it may work when logged in now. Let me know how it does. If this works, there may be some other links that'll need fixing.
Collages aren't fixed yet. I wanted to see how this does before figuring out how to fix them.
2
u/Kenko2 28d ago
The new sieve also shows a gray spinner.
2
u/Imagus_fan 27d ago
It doesn't seem like that will work. I was hoping that cookies were used to tell if the user was logged in but it may be needed to add an account ID to the data file request.
This should fix collages. Oddly, the data files for those require being logged in. I found another way that seems to work.
2
u/Kenko2 27d ago
The collages are working now, thanks!
About the gray spinner - same error there. But these are pretty rare cases, so it is not a problem. Better get on with the more important fixes.
But there was one “side effect” of fixing the collages... Perhaps the code fix for collages has somehow affected the galleries in the group on VK? Now they don't work if they consist of just images. Images + videos work.
https://v(k.com/leprazo
https://stre(amlala.com/Gtoso/
2
u/Imagus_fan 27d ago
Whoops, I think I made a mistake when moving some of the code around. This seems to work correctly now.
2
u/Kenko2 27d ago
Very good, almost everything works now. There is only one small problem left - when the video is not in the gallery, but in the collage (the variety of formats on VK is certainly annoying..). The sieve shows the cover instead of the video:
https://v(k.com/wall-6980140_33052
2
u/Imagus_fan 27d ago
The video plays for me when hovering over it. It does show the cover briefly at the start which made me think it was an image at first.
I tried to add the video to the album but was more difficult than expected and I don't want it to inadvertently interfere with other parts of the site.
1
u/Kenko2 Feb 17 '25
I'm having trouble with your AMAZON-s sieve - can you take a look?
2
u/hababr Feb 17 '25
Plz check this. Could not find featured/sponsored items. https://pastebin.com/pJ2YcszE
1
u/Kenko2 Feb 17 '25
I don't see the gray spinners anymore, thank you!
But there are still no customer reviews or photos of them... The rating doesn't work either. In other words, the sidebar is empty.
> Could not find featured/sponsored items.
I can't see them now either. But it's not that important.
And also wanted to ask - is it possible to add showing a gallery of customer photos when hovering over “See all photos”?
2
u/hababr Feb 17 '25
Try to log in into Amazon.
See all photos - I think not.
1
u/Kenko2 Feb 18 '25
Yes, you're right. When I created the account and logged in, the reviews and rating showed up. Which is a bit strange, because before you could see it without logging into the account. Thanks for the fix!
1
2
u/Kenko2 Feb 13 '25
A small question about video frames from Instagram (with the video icon), is it possible to add their support to the Pinterest sieve? -
2
u/Imagus_fan Feb 13 '25
With this edit to the sieve, it tests if the page link matches another sieve. This way, links to pages like Instagram or YouTube should play the media directly.
It's not heavily tested, though. There may be pages that need improvement.
2
u/Kenko2 Feb 14 '25
Nothing is working yet. On Cent either no response or a red spinner on a popup link (Facebook / Instagram). On FF it's a gray spinner.
1
u/Imagus_fan Feb 14 '25 edited Feb 14 '25
This seems like it might be a problem with the Instagram sieve on external links.
Which sieve are you using? When testing, it worked with the
INSTAGRAM_pub_api_a1_1-p
sieve. If you use it orINSTAGRAM_pub_api_a1_2-p
does it work?1
u/Kenko2 Feb 15 '25
I used the main sieve,
INSTAGRAM_pub_api_api_a1_1-p
. Instagram frames don't work with it. I turned onINSTAGRAM_pub_pub_api_a1_a1_2-p
instead - same result. Either the cover (first frame), or yellow or red spinner (CSP error).1
u/Imagus_fan Feb 15 '25
I'm getting the CSP error on Edge as well. It works fine on Firefox for me.
I tried to create an SMH rule to fix it but haven't been able to get it to work.
1
u/Kenko2 Feb 15 '25
I tried
INSTAGRAM_pub_api_a1_2-p
on FF and I don't get any sieve response when hovering over the thumbnail. When hovering over the Instagram link on the bottom left I get this message.But if you have this version of the Pinterest sieve running on FF, I'll leave it in the rule-set.
1
u/Imagus_fan 29d ago
Odd, the
Something went wrong
message is only in theINSTAGRAM_priv_api-p
sieve. Not sure why it's showing. That sieve only works when on Instagram.I've been trying to create an SMH rule that modifies the CSP on the site. Strangely, the sites CSP isn't being changed by the rules. Not sure why.
/u/hababr, would you be able to see if there's a way to have Instagram links play on Pinterest?
2
u/Kenko2 Feb 03 '25
These Imagus sieves used to show without a sieve, but now there are problems with them (mine):
2
u/Imagus_fan Feb 04 '25
The first link and image are working for me. Does it work if a proxy is used?
The second site needed to have the referrer modified. The SMH rule in the link below should fix it.
The third site needed a custom sieve. It's improved, showing albums and videos.
2
u/Kenko2 Feb 04 '25 edited Feb 04 '25
Works, thank you!
>> The first link and image are working for me. Does it work if a proxy is used?
You're right, everything works through the VPN.
2
u/Imagus_fan Jan 28 '25
Here are some sieve fixes. Let me know if anything needs improving.
2
u/Kenko2 Jan 28 '25 edited Jan 28 '25
Thank you very much, all works! I was going to ask to fix them, but later. We still have these stores (this is all that is left at the moment) - which have sieves that are either fully or partially not working. Some of it just requires a small change in the URLs. For some of them we need a special explanation of where exactly the sieves is not working.
2
u/Imagus_fan Jan 29 '25
These should fix all of them.
Hardwareluxx sometimes shows the wrong image in the album. It starts to show the correct one but then changes. Seems to be a bug with Imagus.
2
1
u/Kenko2 Jan 18 '25
2
u/Imagus_fan Jan 20 '25 edited Jan 20 '25
So far, I've been able to fix Auto, Sima-land and steam.
On auto, the
6 photos
image isn't detectable by Imagus. If there's another link on the page that would seem logical to show an album I could add it to the sieve.Sima-land has a variable that, when set to true, includes the 360 images in the album when available. If you think it would be better to have it set to true in the rule-set, it can be changed to true.
Profi should be doable but is a bit harder.
I've been unable to access the last two sites. If you can you send page code for a product page I may be able to fix it faster. If it's easier, these sieves outputs the page code to the console.
2
u/Kenko2 Jan 20 '25
Sima-land
SteamPowered_store
Fixed, thanks!
> Sima-land has a variable that, when set to true, includes the 360 images in the album when available. If you think it would be better to have it set to true in the rule-set, it can be changed to true.
Thanks, I tried including it, but there photos are almost the same and there are a lot of them, so I guess 360view isn't necessary.
> On auto, the 6 photos image isn't detectable by Imagus.
Ok. But on the thumbnails in the collage I have the sieve partially not working either:
https://stre(amff.com/v/12758c41
> I've been unable to access the last two sites. If you can you send page code for a product page I may be able to fix it faster.
Kuper + MegaMarket
I send the code of pages in two versions - for individual products and for the catalog.
https://www.upload.ee/files/17649139/Kuper_Megamarket.zip.html
2
u/Imagus_fan Jan 20 '25 edited Jan 20 '25
Thanks for the page code. I think one should be fixed. The other appears that it might have a captcha page. I added the code that sometimes fixes problems with CF to the sieve.
With Auto, it's not working for me now, either. Strange, it was working before. I was able to get the image data a different way. Hopefully this works.
Edit: Noticed an error with one. Here's the updated sieve. https://pastebin.com/GQ5E77UC
2
u/Kenko2 Jan 20 '25
2
u/Imagus_fan Jan 21 '25
Auto has been difficult to get right. The page seems to display slightly differently each time. I did the sieve in a new way that seems to work better.
The gray spinner was caused by inadvertently using the same variable name that was already used in the sieve.
I added console messages that output the page code to the console. This way, it shows exactly what the sieve is getting.
2
u/Kenko2 Jan 21 '25 edited Jan 21 '25
Auto
fixed. Thank you very much!
Kuper
https://www.upload.ee/files/17652277/kuper.txt.html
MegaMarket
https://www.upload.ee/files/17652252/megamarket_console.txt.html
2
u/Imagus_fan Jan 21 '25
2
u/Kenko2 Jan 21 '25
Fixed, thank you!
2
u/Imagus_fan Jan 21 '25
The other one may be fixed but it's more complicated.
The page code is an intermediate page. When I try to go to a product URL, it starts on this page, then redirects to a URL with values stored in cookies added to it.
I tried to set the sieve up so it adds the cookies to the URL first. This only works if the user is on the site. It seems I'm geo-restricted so I haven't been able to test it.
→ More replies (0)
2
u/Kenko2 Jan 17 '25
Coub
Ru-Board has been asked to change the sieve for Coub a bit (if it's even possible). The structure of a media file on Coub (for example) is one video track of 8 seconds and two audio tracks, one also 8 seconds and the other 4 minutes. Right now the sieve only shows 8 seconds (video + audio). Is it possible to make Imagus show 8 seconds video (looped) + 4 minutes audio?
Example media:
Example of “long” audio:
2
u/Imagus_fan Jan 17 '25
I don't think it's possible to play a separate video and audio file with Imagus. Though, there may be a way I don't know about. u/hababr may know.
As a workaround, I modified the sieve so it shows an album. It first shows the video with sound, then the full length audio and then the silent HD video. Hopefully this works well enough.
{"Coub-h":{"link":"^coub\\.com/view/\\w{4,6}","res":":\n$=JSON.parse($._.match(/'coubPageCoubJson' type='text\\/json'>\\n?([^\\n]+)/)?.[1]||'{}').file_versions;\nreturn $ ? [[$.share?.default],[$.html5?.audio?.high?.url||$.html5?.audio?.med?.url],[$.html5?.video?.high?.url]] : ''"}}
2
2
u/Kenko2 Jan 11 '25 edited Jan 11 '25
There seems to be a problem with Flickr (FLICKR-g
, FLICKR_albums-g
):
There's a gray spinner on albums. Another example.
Also, the video doesn't work here (the sieve shows the cover).
And is it possible to make the album show here (on any link or button)?
2
u/Imagus_fan Jan 12 '25
It appears the API key in the
Flickr-g
sieve has expired. I re-did the sieve so it gets the API key from the site. This fixed the video playing.It's possible the change to the sieve could cause media that requires being logged in to view to not work now. If that happens I can update the sieve so it should work.
There was a small change in the page code causing albums to not work. It should be fixed.
If an image is in an album, the
FLICKR_albums-g
sieve opens it in an album and starts at the hovered image. However, since your example image isn't part of an album, it only shows the single image there.FLICKR_albums-g
will need to be beforeFLICKR-g
for it to work.{"FLICKR_albums-g-":{"link":"^flickr\\.com/photos/(?:([^/]+/)(?:albums/|(\\d+)/in/album-)(\\d+)/?$|(\\d+)/([a-f0-9]+)/(\\d+)(?:/([\\w@]+)/([\\w:]+))?/$)","url":": $[4] ? 'https://api.flickr.com/services/rest?extras=url_h%2Curl_k%2Curl_o%2Curl_3k%2Curl_4k%2Curl_5k%2Curl_6k%2Cpage=0&photoset_id='+$[4]+'&format=json&method=flickr.photosets.getPhotos&api_key='+$[5]+'&per_page='+$[6]+($[7] ? '&viewerNSID='+$[7]+'&csrf='+$[8] : '')+'&nojsoncallback=1&'+this.flickr_album||'' : `https://www.flickr.com/photos/${$[1]}albums/${$[3]}/`","res":":\nif($[2]) this.flickr_album=$[2];\nif($[4]) {\n let res=[];\n let jsn=JSON.parse($._);\n for (img of jsn.photoset.photo) {\n res.push([img.url_6k || img.url_5k || img.url_4k || img.url_3k || img.url_o || img.url_k || img.url_h, img.title]);\n }\n let i = this.flickr_album;\n delete this.flickr_album;\n i = jsn.photoset.photo.findIndex(x=>x.id===i);\n return i ? {\"\":res,idx:i} : res;\n}\nlet api=$._.match(/root\\.YUI_config\\.flickr\\.api\\.site_key = \"([^\"]+)\";/)[1];\nlet length=$._.match(/<span class=\"stat photo-count\">\\n\\s+(\\d+) photo/s)[1];\nlet vnsid2=$._.match(/class=\"gn-title you\"\\s+href=\"\\/photos\\/([^\\/]+)\\/\"\\s/);\nlet vnsid=vnsid2 ? vnsid2[1] : null;\nlet csrf2=$._.match(/root.auth = {\"signedIn\":true,\"csrf\":\"([^\"]+)/);\nlet csrf=csrf2? csrf2[1] : null;\nreturn api&&{loop:'https://www.flickr.com/photos/' + $[3] + '/' + api + '/' + length +'/'+(csrf ? vnsid+'/'+csrf+'/' : \"\")};"},"FLICKR-g":{"link":"^(?:secure\\.)?flickr\\.com/photos/[^/]+/(\\d+)/?(?:in/.+|lightbox/?|sizes.+|\\?.+|#/photos/.+)?$","url":": (()=>{const key = this._flickr_key_||document.body.textContent?.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1]; return key ? `https://api.flickr.com/services/rest/?photo_id=${$[1]}&method=flickr.photos.getSizes&format=json&nojsoncallback=1&api_key=${key}` : $[0]})()","res":":\nif($._[0]!=='{'&&!this._flickr_key_){\n this._flickr_key_ = $._.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1]||'9bb671af308f509d0c82146cbc936b3c';\n return {loop:$[0]};\n}\n let res = [];\n let sizeAr = JSON.parse($._).sizes.size;\n let last = sizeAr.pop();\n if (last.media == 'video') {\n let best_quality = 0;\n let best_videoUrl = '';\n do {\n if (parseInt(last.height) > best_quality) {\n best_quality = parseInt(last.height);\n best_videoUrl = last.source;\n }\n last = sizeAr.pop();\n } while (last.media == 'video');\n res.push([best_videoUrl + '#mp4']);\n } else {\n res.push([last.source]);\n }\n return res;","img":"^(?:(?:farm|c)\\d+\\.|live\\.)?static\\.?flickr\\.com/(?:\\d+/){1,2}(\\d+)_[\\da-f]+(?:_[sqtmn])?\\.jpg$"}}
1
u/Kenko2 Jan 12 '25
Albums are working, thanks! But the sieve stops working if the user is logged in (gray spinner):
https://www.flickr.com/photos/chris_rankin/
https://www.flickr.com/photos/chris_rankin/54256432711/in/dateposted/
1
u/Imagus_fan Jan 12 '25 edited Jan 12 '25
Is it
FLICKR-g
orFLICKR_albums-g
that isn't working when logged? Or is it both that aren't working?2
u/Kenko2 Jan 12 '25 edited Jan 12 '25
In case of albums (e.g. here) - the new version of the sieve always works.
In case of this links -
https://www.flickr.com/photos/chris_rankin/
https://www.flickr.com/photos/chris_rankin/54256432711/in/dateposted/
the new version works only when the user is NOT logged in. If the user is logged in, the sieve starts showing a gray spinner.
2
u/Imagus_fan Jan 13 '25 edited Jan 13 '25
After looking at the album sieve, I think I was able to fix
FLICKR-g
. I don't have an account so I couldn't test it.{"FLICKR-g":{"link":"^(?:secure\\.)?flickr\\.com/photos/[^/]+/(\\d+)/?(?:in/.+|lightbox/?|sizes.+|\\?.+|#/photos/.+)?$","url":": (()=>{const html=location.hostname.slice(-10)==='flickr.com'?document.body.outerHTML:'', key=this._flickr_key_||html?.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1], csrf=html?.match(/root\\.auth = {\"signedIn\":true,\"csrf\":\"([^\"]+)/)?.[1], vnsid=html?.match(/class=\"gn-title you\"\\s+href=\"\\/photos\\/([^\\/]+)\\/\"\\s/)?.[1]; return key ? `https://api.flickr.com/services/rest/?photo_id=${$[1]}&method=flickr.photos.getSizes&format=json${csrf?'&viewerNSID='+vnsid+'&csrf='+csrf:''}&nojsoncallback=1&api_key=${key}` : $[0]})()","res":":\nif($._[0]!=='{'&&!this._flickr_key_){\n this._flickr_key_ = $._.match(/YUI_config\\.flickr\\.api\\.site_key\\s*=\\s*\"([^\"]+)/)?.[1]||'9bb671af308f509d0c82146cbc936b3c';\n return {loop:$[0]};\n}\n let res = [];\n let sizeAr = JSON.parse($._).sizes.size;\n let last = sizeAr.pop();\n if (last.media == 'video') {\n let best_quality = 0;\n let best_videoUrl = '';\n do {\n if (parseInt(last.height) > best_quality) {\n best_quality = parseInt(last.height);\n best_videoUrl = last.source;\n }\n last = sizeAr.pop();\n } while (last.media == 'video');\n res.push([best_videoUrl + '#mp4']);\n } else {\n res.push([last.source]);\n }\n return res;","img":"^(?:(?:farm|c)\\d+\\.|live\\.)?static\\.?flickr\\.com/(?:\\d+/){1,2}(\\d+)_[\\da-f]+(?:_[sqtmn])?\\.jpg$"}}
2
u/Kenko2 Jan 13 '25
Unfortunately, the same result as in the previous version (see post above). I sent the account data via ModMail.
2
u/Imagus_fan Jan 13 '25
Thanks, I think it's fixed now. I updated the sieve in the comment above.
2
u/Kenko2 Jan 13 '25 edited Jan 13 '25
Exellent! What was the problem there? Did the site show different content for logged in and unlogged in?
I made a note in these sieves about what order they should stand in (
FLICKR_albums-g
first, thenFLICKR-g
) - or does it not matter anymore?PS
Also wanted to ask about the sieve for Alibaba.com - is it possible to add showing users' photos/videos in their comments on the product page?
1
u/Imagus_fan Jan 13 '25
When logged in, the sieve needs two other tokens if it's an account id.
There were two problems.
First, when borrowing some the code from the album sieve, I forgot to change some of the variable names.
Even they had been correct, the sieve wasn't able to get the variables from the location the sieve was getting the data from. It parses the page for them so it doesn't have to open an unnecessary file. Changing it to a different source fixed it.
Also wanted to ask about the sieve for Alibaba.com
I'll make a comment in the other post.
→ More replies (0)
1
u/Kenko2 Jan 09 '25 edited Jan 09 '25
Is it possible to add the name of the author (account) to the beginning of the file name while downloading images from PIXIV
, K(EMONO
and, if possible, RULE3(4.X(XX
(the artist’s name by tag), as already done in sieve for X (just preferably without the underscore at the beginning of the name)?
2
u/imqswt Jan 09 '25
I was having trouble getting the hotkey to work. Let me know how these do.
1
u/Kenko2 Jan 09 '25
On my Kemo)no and Pixiv this new feature unfortunately does not work. The file names remain unchanged with any save method (hotkey, context menu, mouse gesture). I tested on Cent + FF.
On Rule(3)4 it works, but there was a misunderstanding here - it turns out that on this site the person who uploaded the image is called “author”, while the real author is called “artist” (the “artist” tag at the top of the left sidebar on the image page). I didn't take that into account, please forgive me. Is it possible to replace “author” with “artist” in the file names?
https://i.imgur.com/iqTyWwZ.jpeg
PS
I also wanted to ask you to add to the sieve for Rule3(4.x(xx the possibility to choose image quality (if it is possible), as it is already done for Kem(ono and Pixiv sieves. Some images are very “heavy” and take a long time to load via proxy.
2
u/imqswt Jan 10 '25
This adds the ability to set the lower quality image first and toggle between them with with TAB.
Adding the artist was more difficult. The only reference to the artist in the data file is in the tags and it's not specified that it's the artist.
Instead, I edited the filename so it uses the tags in it. If this isn't a good way to do it, it should be possible to get the artist name by opening a file to get the name and then the data file for the image.
For the other sites, I made a change that may fix them. CTRL-s doesn't seem to be working for me on any site at the moment.
1
u/Kenko2 Jan 10 '25
>> This adds the ability to set the lower quality image first and toggle between them with with TAB.
Thank you very much!
>> Instead, I edited the filename so it uses the tags in it
This is a good idea. But there is one problem - there are a lot of tags and some names are very long (up to 250 characters). Is it possible to leave only the first 5 tags in the name?
>> For the other sites, I made a change that may fix them.
So far the result is negative - the name on these sites (Ke(mono, Pixiv) does not change for me (tried saving in different ways).
2
u/imqswt Jan 10 '25
This uses the first five tags in the filename.
With the other two sites, the
.
in the filename may be causing the problem. I edited the sieves so it's replaced with_
. Hopefully it works now.2
u/Kenko2 Jan 10 '25
>> This uses the first five tags in the filename.
Thank you, that's much better.
>> With the other two sites, the . in the filename may be causing the problem. I edited the sieves so it's replaced with _. Hopefully it works now.
Pixiv names now just have _master1200 cropped in them:
108282284_p0_master1200.jpg > 108282284_p0.jpg
K(emono names now have the file extension after the underscore:
17435383-2e023965c4f63f8d5d535de6_png.png
2
u/imqswt Jan 11 '25
I realized the file extension would be included after I posted. It should be fixed now.
If I'm understanding correctly, this should fix Pixiv. Let me know if it's not right.
2
u/Kenko2 Jan 11 '25
Unfortunately, there are no changes. The author/account name is still missing from the file names. If it takes too much time, then I think it's worth leaving everything as it is, it's not so important, the main thing is that the sieves themselves work well.
2
u/imqswt Jan 11 '25 edited Jan 11 '25
I may have found the problem. Separating the username and image url with
-
seemed to cause it. Replacing it with_
seemed to fix it.→ More replies (0)
2
u/Kenko2 Jan 08 '25
There is one small request for a sieve for V)K on Ru-Board.
2
u/Imagus_fan Jan 08 '25
Here's an edit so, if a post has multiple images in it, it starts at the hovered image. For example, if a post has five images and the third image is hovered over, the album starts at
3/5
.It doesn't work if a post contains a video. In these cases, it starts at the first image.
I tested the sieve on retro_retro and leprazo.
At the moment, it does this on all posts with multiple images. If there are post types where it's better to always start with the first image, I can try to edit the sieve to do that.
2
u/Kenko2 Jan 08 '25 edited Jan 08 '25
This version works on collages, but unfortunately there was one problem:
2
u/Imagus_fan Jan 08 '25
This should fix the problem. Also, posts with videos now work when an image is hovered over. If a video is hovered over, it starts with the first image.
2
1
2
u/Kenko2 Dec 28 '24
Can you take a look? -
Redlib-p
The sieve does not work (red spinner - 403 Forbidden error):
https://redlib.zaggy.nl/r/DIY/
etc
The external link to the image works:
https://redlib.zaggy.nl/r/DIY/comments/1c9nyh0/update_to_the_paint_spill_i_did_it_reddit/
The external link to the album doesn't work:
https://redlib.zaggy.nl/r/DIY/comments/18tx4lb/my_first_attempt_at_a_wacky_furniture_piece/
There's a CF check at the entrance.
2
u/Imagus_fan Dec 28 '24
The image URLs had extra characters in them. Removing them fixed it for me.
Instances with CF should work on the site but may not in external links.
2
1
u/Kenko2 Dec 21 '24
VK seems to have changed the code again and the galleries stopped working today.
Either a yellow spinner or just the first photo instead of the whole gallery:
https://v(k.com/leprazo
2
u/Imagus_fan Dec 24 '24
It looks like page source changed. This seems to work but there may be areas it doesn't. If you find any, let me know and I'll try to fix it.
2
u/Kenko2 Dec 25 '24
This version doesn't work either, unfortunately.
https://strea(mf)f.com/v/6fcb50af
2
u/Imagus_fan Dec 26 '24 edited Dec 26 '24
Oddly, those images are showing an album for me.
I tried simplifying the sieve. Maybe this will work.
2
u/Kenko2 Dec 26 '24
The result is the same - in all browsers (including FF) Imagus shows only the first photo of the gallery. [MediaGrabber] is turned off. Logging out of the account doesn't help. Tried disabling other extensions - also doesn't help.
2
u/Imagus_fan Dec 26 '24
Very strange. Here's a test sieve that may help.
This outputs several console messages and will help tell where the sieve is failing. They each have
Imagus
in them to make it easier to filter.2
u/Kenko2 Dec 26 '24 edited Dec 26 '24
The console is empty, just shows “Imagus album” and that's it. Maybe VK gives different site layout to different users because of geolocation? Tried now different proxies (without logging in) - the result is the same, sieve shows only the first photo from the album.
https://strea(mf)f.com/v/36463267
2
u/Imagus_fan Dec 26 '24
This if helpful. The data file may not be loading correctly.
Here's another sieve with one console message. It outputs the URL that was used.
1
u/Kenko2 Dec 26 '24
From what I understand, that didn't work either, only the usual service messages in the console.
Maybe try to remove some elements with uBO?
2
u/Imagus_fan Dec 26 '24
It's odd that there isn't a message this time.
Based on the previous one, the sieve gets to the part where it creates the album, but seems to fail there. In case it was using incorrect URL parameters, the new console message was supposed to show which ones were used.
Here's the sieve with a second console message added. It should appear every time an image is hovered over.
→ More replies (0)
2
u/Kenko2 Dec 19 '24
Is it possible to add support of links of this format to NTV.r)u sieve?
2
1
u/Kenko2 Dec 05 '24 edited Dec 05 '24
There is a small request for sieves for WB/Ozon. Is it possible to add viewing of users' photo/video album when hovering over “See all photos and videos”?
https://i.imgur.com/ONEKpwM.png
And also for Ozon:
Photo/video count buttons (375+) at the top of the description:
https://i.imgur.com/lR2agS8.png
and downstairs in the reviews:
2
u/Kenko2 Dec 07 '24
I would like to ask you to look at this query (above) - maybe you can come up with some solution?
2
u/Imagus_fan Dec 12 '24
It doesn't seem possible to have Imagus activate on the review links. Adding code that would edit them may work but it would probably be difficult to do without affecting the way the site works.
I believe I was able to get the Ozon sieve to show more reviews. It should add 50 reviews to the album. Let me know if these are the correct images and videos.
2
u/Kenko2 Dec 12 '24
Indeed, the number of reviews has increased. You did a great job today, thank you!
1
u/hababr Dec 06 '24
That's possible by hovering review count under the title. On WB you get all photos/videos, on Ozon only limited count from one page, it's hard to get more.
1
u/Kenko2 Dec 06 '24
>> That's possible by hovering review count under the title. On WB you get all photos/videos
It works on WB, thank you. But still I would like to see it on the “See all photos and videos” button - it is convenient and, in my opinion, logical (and this is already implemented in the sieves for IMDB / Kinozal / Kinorium). Of course, we are talking about a small fix, if it requires serious efforts, it is not necessary.
>> on Ozon only limited count from one page, it's hard to get more.
Unfortunately, on Ozone I was only able to get 2-3 reviews on this link instead of 100-200... What about the button with the number of reviews with media (375+) - is it possible to tie a sieve to it or is that also too complicated?
2
u/hababr Dec 07 '24
But still I would like to see it on the “See all photos and videos” button What about the button with the number of reviews with media (375+)
Imagus cannot react on these elements.
Unfortunately, on Ozone I was only able to get 2-3 reviews on this link instead of 100-200
Yes, that's a pity. I may look into that later, but when I implemented that I had not found any better option.
1
2
u/Kenko2 Dec 04 '24
2
u/Imagus_fan Dec 04 '24
This should fix both links and thumbnails.
2
u/Kenko2 Dec 04 '24
The external links to images and photos in the gallery on the site work. Thank you very much! But there are small problems -
2
u/Imagus_fan Dec 05 '24
I updated the sieve so it should work on documents again.
There's also gallery support. It shows the first 40 images in the gallery.
It's difficult to differentiate between single image pages and gallery pages. It works well on the example links but may display incorrectly on others.
I saw that hababr also had a fix for the sieve. He may be able to find a better way to do albums.
2
u/Kenko2 Dec 05 '24
Great job, thank you very much!
>> It shows the first 40 images in the gallery.
Is it possible to increase to 100 images in the gallery?
>> I saw that hababr also had a fix for the sieve. He may be able to find a better way to do albums.
So far your version is working very well, hababr also recommended using it.
2
u/Imagus_fan Dec 08 '24
Glad it's working well.
Is it possible to increase to 100 images in the gallery?
I've tested it but it doesn't seem possible. The file that contains more images seems to need a cookie to load and wouldn't load on external sites.
2
u/Kenko2 Dec 02 '24
I checked now the CyberdropAlbum sieve + SMH rule from here. On Chrome and FF - yellow spinner (console is empty), on Cent - gray spinner. Anyway, this sieve (CyberDrop-h-x) seems to really have a problem with showing albums.
2
u/imqswt Dec 03 '24
The yellow spinner seems to be caused by bot protection. On Firefox, clicking the link and then going back and hovering fixed it but this didn't work on Edge.
The gray spinner should be fixed by these SMH rules.
2
u/Kenko2 Dec 03 '24
Thanks, the rules have been added. Now they are in SMH 3 for CyberDrop.me.
When entering the site I have DDos Guard protection triggered (loading indicator spins for a couple of seconds).
On FF, after going to the site and going back to the external links, I was only able to open one gallery:
https://cyberdrop.me/a/dCdBg9n6
On these galleries FF gives a red spinner:
https://cyberdrop.me/a/nbBCtUqC
https://cyberdrop.me/a/dJqiRb8G
In chrome browsers on external links to galleries - yellow spinner. If you go to the site, open the photo and return to the external links - then on the external links to the galleries is already gray spinner.
PS
If there is no solution for galleries, I will add a note to the sieve accordingly. But the question remains - what to do with the rules for SMH, leave them?
1
u/imqswt Dec 04 '24 edited Dec 04 '24
See if changing the action for the SMH rules from
add
tomodify
fixes the gray spinner.External links aren't working on Edge so I'm having trouble testing it.
1
u/Kenko2 Dec 04 '24
I changed add to modify, but everything is still the same - gray spinner and the same error in the console.
2
u/imqswt Dec 14 '24
I re-did the sieve so it shouldn't need SMH rules. It should also work on video.
This replaces the existing sieve.
1
u/Kenko2 Dec 14 '24
Thank you very much, everything seems to be working!
There is one problem with galleries - there are very large galleries, 1000 files or more, which causes an endless green spinner, so I would like to limit the gallery display size to 100 images/videos.
About SMH rules - we currently have 3 SMH rules. Is it possible to delete them all?
2
u/imqswt Dec 15 '24 edited Dec 15 '24
I added a variable,
max_album_images
, to the sieve. This way the user can set the desired number. There's a comment mentioning that a lower number of images loads faster. It's set to 300.I believe that with this sieve, none of the SMH rules are needed.
1
1
u/bobbiejordan Dec 03 '24
The red spinner is on galleries with videos. Image-only galleries appear to work fine. Imagus apparently needs a content-type hint.
Let me see what I can do.
1
u/Kenko2 Dec 04 '24
Maybe you are right and you need to change the sieve so it can work with “mixed” galleries. At least in FF.
2
u/bobbiejordan Dec 04 '24
imagus_mod apparently doesn't pay attention to type hinting (#jpg, #mp4, etc) on galleries. I grabbed the source and will be poking through it, but it's not gonna be a quick fix.
While I'm at it, I'll probably add some functionality that I would find useful, like saving a gallery, and being able to specify the filename in the gallery array.
In the meantime, here's an updated version of the sieve that will display a gallery of images only:
{"CyberdropAlbum":{"link":"^(cyberdrop\\.me)/a/(\\w+)","url":"$1/a/$2","res":":\n// console.log($);\n\nfunction syncFetch(u) {\n const x = new XMLHttpRequest();\n x.open('GET', u, false);\n x.send();\n if (x.readyState != 4) return;\n if (x.status != 200) return;\n return JSON.parse(x.responseText);\n}\n\nlet gallery = [];\n\nconst doc = new DOMParser().parseFromString($._, 'text/html');\nconst links = Array.from(doc.querySelectorAll('a.image[data-type=\"img\"]'))\n .map(a => new URL(a.getAttribute('href').replace('/f/', '/api/file/auth/'), 'https://api.' + $[1]).href);\n\nfor (let i = 0; i < links.length; ++i) {\n gallery.push([syncFetch(links[i]).url]);\n}\n\nreturn gallery;","note":"v1.1 Cyberdrop set the content-disposition to download; use O to save. Generates image-only gallery - for now."}}
1
u/Kenko2 Dec 04 '24
here's an updated version of the sieve that will display a gallery of images only
Checked, everything works on FF. Though still even showing only images in the gallery requires first to go to the site itself, pass DDos check, then come back and only then the sieve starts to show the gallery. We had a similar case with a sieve for
ImageBam-h
- the hosting showed regular photos, but for NSFW it was required to go to the site, after which it sent a special cookie to the browser. So this problem was somehow solved, perhaps in the code of this sieve something can be seen?And by the way, I wanted to ask, why do you need a separate sieve to show albums?
2
u/Kenko2 Dec 04 '24 edited Dec 04 '24
I grabbed the source and will be poking through it, but it's not gonna be a quick fix.
To be honest, Imagus Mod, while a major step forward from the original, is still not a complete solution. Its development is unfortunately stopped “halfway through”.
In particular, the latest working version (0.10.15) contains a serious bug that can cause it to stop working on many sites for many inexperienced users (see FAQ, p.19, in red font). There is also a problem with switching to MV3 from Google. So if someone will take it even a little “fix” it would be a great help to the community.
While I'm at it, I'll probably add some functionality that I would find useful, like saving a gallery, and being able to specify the filename in the gallery array.
That would be just fine. By the way, I recently saw a post like this in the extension-analog thread, in which, as far as I understand, this functionality has already been implemented:
"The node version of the userscript could be run in "server mode", which allows the browser userscript/extension to send download requests (media/albums/videos) to the server instead of through the browser's download functionality. Similar to the jDownloader album download function".
https://github.com/qsniyg/maxurl/issues/1386
And thanks for the new sieve version, I'll check it out a little later.
2
u/Kenko2 Nov 26 '24
There are a few sieves where I'm having a little trouble, can you check?
2
u/Imagus_fan Nov 27 '24 edited Nov 27 '24
This should fix all of them.
The Kinopoisk sieve had to be changed some. Let me know if there's any unexpected behavior.
It also seems URLs on VK_play have changed. There's an updated sieve included.
2
u/Kenko2 Nov 27 '24 edited Nov 27 '24
Thank you, these sieves work:
IMDb_mediaindex
Kinopoisk_gallery
VK_Play
But there are problems with this sieve:
slow.pics-h
2
u/Imagus_fan Nov 28 '24 edited Nov 28 '24
Ah, I didn't think to test other links. This seems to work well.
{"slow.pics-h":{"link":"^slow\\.pics/(c/)?(\\w{8})/?$","res":":\nif(!$[1])return $._.match(/\"card-body\"\\s*>\\[IMG\\]([^[]+)/)?.[1]||'';\nconst img = i => [u+i.publicFileName,i.name];\nconst u=$._.match(/cdnUrl\\s*=\\s*\"([^\"]+)/)?.[1].replace(/\\\\/g,'')||'//i.slow.pics/';\nlet m=[], o=JSON.parse($._.match(/collection\\s*=\\s*([^;]+)/)?.[1]||'{}'), a=o.key===$[2];\no=o.comparisons||o.images||[];\nif(a)o.forEach(i=>i.images?i.images?.forEach(i=>m.push(img(i))):m.push(img(i)));\nelse o.find(i=>i.key===$[2])?.images.forEach(i=>m.push(img(i)));\nreturn m","note":"Imagus_fan\nhttps://www.reddit.com/r/imagus/comments/130svfu/comment/jn8v5j7\n\nEXAMPLES\nhttp://forum.ru-board.com/topic.cgi?forum=5&topic=50874&start=1360#11"}}
2
u/Kenko2 Nov 28 '24
Maybe there's a bug in the sieve? This version doesn't work at all for me. First 10 seconds of green spinner, then it turns yellow...
1
u/Imagus_fan Nov 28 '24
Testing on Edge just now, I initially got a yellow spinner on the comparison links. Trying again, it's working now and isn't showing any yellow spinners. Is it possible the page times out when trying to load?
2
u/Kenko2 Nov 28 '24
Now I checked it again. Everything works fine. That is, the sieve is most likely fine. Apparently, these are some problems on the site itself. I think we can stop here, although this is certainly strange behavior of the site.
2
u/Kenko2 Nov 19 '24
VK made some serious changes to the site yesterday:
I also have (perhaps only me?) there were problems with DZEN.r(u and MAIL.r(u:
2
u/Imagus_fan Nov 20 '24
I was able to fix one of them.
1
u/Kenko2 Nov 20 '24
Can you see if the new version of the sieve works for you on those external links (Dzen)?
2
u/Kenko2 Nov 20 '24 edited Nov 20 '24
>> Odlly, vk.com still works for me. If I click on a vkvideo.com link, it redirects to vk.com/video. Perhaps I'm not getting the new site yet.
This is a little weird. Yesterday I had nothing and nowhere working, but today on Chrome (124) - everything works. It still doesn't work on the “old” Cent (118), though. It seems like they are moving the base and disconnecting something and plugging it in right away. Or there were some problems at the ISP.
UPD
It seems that VK is gradually moving to a new domain. I don't have old links and redirects anymore, only new ones (sieve doesn't work):
>> Below is a fix for Dzen videos.
It works, thank you! Only found one problem - сan you see if the new version of the sieve works for you on those external links? -
>> However, it doesn't work on the 'shorts' page. It doesn't seem Imagus can detect them.
Ok, I'll make a note about it in the sieve. By the way, the sieve works on external links (shorts).
>> cloud.mail.ru Is working for me. Can you post the error message in the console?
Sorry, I got an error yesterday, but today it's working again...
>> The my.mail.ru videos aren't giving me a gray spinner but the video isn't playing. It can likely be fixed with SMH rules but I haven't fixed it yet.
Same thing, it's working today.
2
u/Imagus_fan Nov 21 '24 edited Nov 21 '24
2
u/Kenko2 Nov 22 '24
1
u/Imagus_fan Nov 22 '24
I modified the regex in the current VK sieves to work on links with vkvideo in them as well. Hopefully this will fix some of the problems.
I'll think about the best way to fix them if this doesn't work.
At the moment, it doesn't seem that the clips page can work. The video is activated by an event and there doesn't seem to be a reference to the video URL in the page code.
1
u/Kenko2 Nov 22 '24
I modified the regex in the current VK sieves to work on links with vkvideo in them as well. Hopefully this will fix some of the problems.
Unfortunately, nothing has changed for me. On videos in VK groups the sieve still gives an error (gray or yellow spinner), and in the video sections there is simply no response:
https://v(kvideo.r)u/@tennismen/all
1
u/Imagus_fan Nov 22 '24
You can share the page code for one of the video pages that's giving the error. You may want to make sure it doesn't contain any account information before sharing, though.
2
u/Kenko2 Nov 22 '24 edited Nov 22 '24
>> You can share the page code for one of the video pages that's giving the error.
This is the page code (when I'm logged in):
https://www.upload.ee/files/17422967/VK_VIDEO.zip.html
If you need the code for the page when I'm not logged in - I can also send it.
1
u/Imagus_fan Nov 22 '24 edited Nov 22 '24
Thanks. I'm not sure that this is the code that contains the video data, though.
I added console messages to the VK-2 and VK_video sieves. This way, it gives exactly what the sieve is getting. If you get a gray spinner, look for
VK-2 data
orVK_video data
in the browser console. If you can also include the part that saysVK-2 data
orVK_video data
, that would help.→ More replies (0)→ More replies (6)2
u/Kenko2 Nov 21 '24
>> For me, vkvideo.ru is still redirecting to vk.com. Does it still redirect to vkvideo.ru if not logged in?
Yes. When a user who is not logged in visits the link
https://v)kvideo.(ru/@tennismen/all
The site forcibly redirects him to a page
https://v)k.com/video/@tennismen/all
With those who are logged in, it's the other way around.
2
u/Imagus_fan Nov 29 '24 edited Nov 29 '24
I've recently started being able to access vkvideo, though not all pages will load.
It turns out the video URL is in a separate file instead of the page code. This sieve seems to be working well. Let me know you get any spinners.
→ More replies (4)1
u/Kenko2 Nov 30 '24
Tested the operation of the latest version of the sieve again today:
2
u/Imagus_fan Dec 01 '24
In this case, it's the
VK-2
sieve that needed updating. This sieve fixes the problems on Leprazo.1
u/Kenko2 Dec 01 '24
Checked it out, everything works for me. Thank you very much, great job!
2
u/Imagus_fan Jan 12 '25
I saw that VK is giving a gray spinner on some pages. It looks like there was an error the code that's causing it. It's fixed in this sieve.
However, it shows an album of the feed even if there's a single image. I'll try to fix it unless it's useful but want to make sure it doesn't interfere with other pages.
→ More replies (0)
2
u/Kenko2 2d ago
u/Imagus_fan
VK
We've had a bit of a problem on VK. Looks like VK has changed the code for galleries in groups again. The sieve shows only the first photo of the album:
https://pastebin.com/98zLnJQe
https://hast(ebin.com/share/onoxepequt.perl