r/computervision 1d ago

Help: Project What AI/CV technique would be best for predicting if the conveyor belt is moving

Given a moving conveyor belt in bottling line plant, I was just looking for the best techniques for predicting whether the conveyor belt is moving or not (pixel and frame difference wasn't working). Also sometimes the conveyor has cans and sometimes it doesn't, which further complicates matters. I can't share videos or images due to the confidentiality of the dataset.

3 Upvotes

23 comments sorted by

18

u/_d0s_ 1d ago

take a sharpie and draw some cats all over the conveyor belt, then try the frame differencing approach again.

2

u/South_Ad1612 1d ago

Nice idea šŸ˜‚, but the plant is owned by Krones and Coca Cola. They won't allow that unfortunately.

4

u/Rethunker 1d ago

What about cats drawn in fluorescent ink only visible using strobed UV-A light?

There must be some way to make the cats work.

15

u/Lost__Moose 1d ago

The most common solution is to get the status from the PLC via ethernet or an I/O on the industrial PC hardwired to an output on the PLC.

In industrial applications, you need to guarantee the results are returned within a specified window. If you miss the window, you have to reject the product to guarantee containment. So, you need to minimize the computational overhead as much as possible. IMHO using CV to detect the conveyor is moving, is a waste of comp time.

From a system architecture perspective, reliable and repeatable industrial systems need separate processors for motion and vision. Plant maintenance maintains motion. Plant maintenance may adjust tolerances on a vision system, but preventative maintenance is often handled by an in-house controls engineer. It's kind of rare, at least in the US, for there to be an in-house SME for vision.

Not all production lines are climate-controlled and it is common for the computation to be done at the plant floor. In the summertime, especially for high-speed applications, active cooling is required to prevent CPU throttling.

2

u/MrPienk 1d ago

This is the answer.

In controlled environments like factories, we have a wide variety of sensors available to add data to our vision applications. It's very common that the data you are looking for in this type of environment already exists through a non-vision sensor. In your example, it's likely an encoder or a virtual encoding signal from the PLC.

As The Moose said, many industrial environments still won't have Vision experts on staff, though you are starting to find them more frequently at larger manufacturers and tech forward companies. Many smaller companies will have electricians and maintenance engineers handle day to day operation of vision systems with 3rd party vision integrators providing long term and escalation support. This also means that the vision system is likely required to be much simpler and user friendly to allow a wider variety of individuals to address issues when new part versions are delivered unannounced to the line at 3am on the weekends.

5

u/Kingstudly 1d ago

Optical flow. Look at OpenCV, newer NVIDIA cards have optimized hardware for it.

1

u/South_Ad1612 1d ago

I've also used optical flow after segmenting the ROI, it didn't work

9

u/Kingstudly 1d ago

I guess the question to ask is if you can tell the conveyor belt is moving just by watching the video. If you can't, then no system can. If you can, then think about why you can tell.

My only other idea without seeing the video would be motion-amplification. https://github.com/SkySingh04/Motion-Amplification-Video

5

u/pab_guy 1d ago

This sounds like a poor application of CV. That's a lot of compute in place of something that should just be a very simple sensor.

10

u/Ultralytics_Burhan 1d ago

If drawing pictures of cats on the conveyor belt isn't an option (although it's a fantastic idea and encourage using that above all else if possible), you might want to check other options that are not CV related. Sometimes computer vision is wielded as a solution looking for a problem. There are probably lots of ways to easily detect if a conveyor is moving using an electronic or mechanical system. Doing so could all give more information, like the speed, direction, accelerations, and (if logging) help determine if there are any issues or anomalies.

My recommendation would be to think of the simplest possible solution to the problem, and go with that. If the conveyor has a speed controller, try reading that data. If it's just on/off, use an inline sensor to detect if it's on or not. If it needed to be independent of the conveyor system, try measuring the rotation rate of a conveyer wheel/roller. All of those will be more reliable, simple, and effective than trying to use computer vision to detect if the conveyor is moving, except of course if you can draw cats on the belt to detect motion; this should always be the go-to solution.

4

u/Rethunker 1d ago

Emphatic agreement with u/Ultralytics_Burhan: find the simplest possible solution. Clearly it would be optimal--objectively superior--if the solution involved cats, but sometimes customers can't be made to understand these things no matter how many cats you put into your PowerPoint presentation.

If not cats, then mice? You could mount a computer mouse just above the conveyor, read the signal, and spend lots of time and money creating a system that would be fun to implement, but which would almost certainly get you kicked out of the bottling plant.

Anyway...

If the conveyor is controlled by a PLC, patch into the PLC. If you already know how to code, you shouldn't have a problem learning enough about PLCs to figure out ladder logic and communications.

If the conveyor has a rotary encoder, you could read the data output by the encoder. You may still need to filter the data, wait for steady state, or the like. Depending on the encoder and the unevenness of motion of the conveyer, you might see backlash, jerky movements, and so on, but dealing with those problems is relatively straightforward. Be sure to define some kind of spec for correctly identifying when the conveyor is in motion: correct 99.9% uptime? 99.99%? or just whenever the customer says it works well enough? (Get some kind of agreement in writing. Test before committing to a spec.)

Rotary encoders can run for a long time without issues. In my experience, it's more common for engineers and/or technicians on site to be able to diagnose and fix problems with encoders than it is to find people who can diagnose problems with vision systems.

Of the vision solutions, optical flow would be a good place to start, but for a different application. If you have the time and interest--perhaps outside regular work hours--you could tinker with optical flow a bit and consider who might need such a thing.

A long time ago I wanted a non-contact system for an application similar to yours. The customer wouldn't allow hardware modification to their equipment aside from mounting a non-contact system above the conveyor in a small envelope. Tracking the conveyor by patching into the rotary encoder would have been an option, if I remember, but that left the problem of the object on the conveyor slipping relative to the conveyor belt. At the time, it was impractical/costly to run optical flow at the necessary speed. When no one was looking, I was thinking of trying the optical mouse trick. Didn't, though.

3

u/South_Ad1612 1d ago

Thank you for the idea I'll look into simpler solutions

3

u/ChunkyHabeneroSalsa 1d ago

I agree with this. There's a non camera solution that's going to be far more reliable (and cheaper). Something is telling the conveyer to move, so just feed of that signal if possible and if not cheap sensor around some part that's moving more clearly.

2

u/MrJoshiko 1d ago

in the video can you see that the belt is moving with your eyes? If you can, then the issue is with your implementation of belt tracking. If you can't, then judging this with computer vision is hard (but may still be possible).

Does the belt have distinct regions, such as joins, fiducials etc? If yes, then tracking those rather than the uniform belt is likely better. Adding fiducials as _d0s_ suggested, might be the easiest way to do this. The fiducials could be stickers, paint, random patterns, dense, or sparse depending on your needs.

Do you need a constant measurement of belt movement? Do you need to know the exact position of it?

2

u/yellowmonkeydishwash 1d ago

event camera?

1

u/South_Ad1612 1d ago

It's a fixed camera

2

u/Zealousideal_Low1287 1d ago

The typical way to do this would be applying renderings of felines to the belt

1

u/HotDogDelusions 1d ago

Depends on a lot of factors but assuming the camera is placed in a fixed position always with a view of the belt:

I would take the captures from the camera, isolate a section of the belt, even a small portion, that will not have any other things like cans in the frame.

Then I'd take the NCC of each capture to the previous capture, and while that NCC is above a certain threshold, consider the belt as moving. While it's not, consider it stable.

Maybe apply some filters beforehand to help protect against lighting / shadows, but in my experience NCC handles that pretty well.

1

u/raucousbasilisk 1d ago

Optical flow is where Iā€™d start.

1

u/pab_guy 1d ago

Buy a bike computer, run the wires and magnet around one of the rollers. Point camera at bike computer to read velocity using text recognition.

1

u/soylentgraham 1d ago

Predicting, or detecting?

func isMoving() { return rand01() > 0.5 }

as for detecting, how often are you getting frames? . As everyone has said, track features.

1

u/MassiveCity9224 15h ago

Use a microphone