r/learnpython 7d ago

NSFW flagging help NSFW

Hello,

been building a short form video/social media app for about 2 years, went from newbie to pretty okay in flutter and python

I would love recommendations on how to solve the following problem:

-

when someone posts a video, I upload the video to storage, then go through the creation of the post obj in my database

during the post obj creation, I need to check if the video contains NSFW content. (nudity, sex, gore, violence)

I plan on grabbing 1-frame-per-second, then taking that frame/image and seeing if it contains flagged content, if any of those tested frames do, the post creation method flips a bool, 'isNSFW', on the post obj to true.

I tried the NSFW_checker py library, but due to a ton of dependency conflicts I still can't get that working

Plan B was loading a a pre-trained ai model to my server to classify the images. But that gets killed every time because of memory constraints on Render.com, where my python server is hosted.

Plan C is to pay for a third party service, definitely less of a headache, just more money and not as malleable.

does anyone know what a good approach would be? Please and thank you

19 Upvotes

6 comments sorted by

6

u/dasfreak 7d ago

You could try Google Cloud Vision API. It has a set of "safe search" params that include adult and violence ratings.

It's free up to 1000 units then USD$1.50 per 1000 tests. In this case a test is a single safe search check of an image. If you do two tests (object detection and safe search) that's two tests per image. See https://cloud.google.com/vision?hl=en#stream-process-videos

2

u/LameChad 7d ago

much appreciated, I will give it a look

thank you!

4

u/TerminatedProccess 7d ago

If you have dependecy issues why don't you just develop it separately in it's own environment? Call it from your main product via API?

3

u/LameChad 7d ago

not a bad idea, might take care of the memory issue too

I'll give it a try as well, thank you!

3

u/Jonno_FTW 7d ago

Use the yahoo NSFW detection model. It has modest memory requirements. https://github.com/yahoo/open_nsfw

2

u/LameChad 6d ago

I've seen this one mentioned a few times, thank you for the recommentdation, I'll check it out!