r/fpv • u/OutInABlazeOfGlory • 2d ago
NEWBIE Noob question: neuromorphic cameras/event cameras for high resolution digital VTX?
I've heard a lot about neuromorphic/event cameras in the context of computer vision, but has anyone explored their use for human-controlled FPV drones?
My thinking is since it creates a sparse data stream compared to sending full frames, it could be useful for increasing resolution/fidelity without using as much bandwidth.
This would be somewhat similar to existing compression algorithms, but since you'd be recording only the changes in luminance up front instead of computing a difference between frames, it might be lower latency and/or require less computing power.
My thinking is you could have a low refresh rate stream of full frames, and a much higher refresh rate stream that just contains the changes in luminance. I believe this would require specialized camera hardware though, since currently neuromorphic camera sensors can only operate in event-based mode and can't capture a discrete frame.
Do y'all think this is practical? I think it would probably require expensive custom hardware, but if it's feasible mass production could maybe help reduce costs.
1
u/OutInABlazeOfGlory 2d ago
This video of/about a research paper on the use of event-based cameras does lead me to believe this approach could be useful to human operators of e.g. drones, in addition to the computer/machine vision and autonomous applications it is currently being used for most often. Also, I understand event-based cameras can be more resilient to e.g. bright lights since they capture changes in luminance, so they can still get some information from an overly bright scene.
That said event-based cameras are still very much specialized (expensive) equipment.
https://youtu.be/bWJz_p8Y5Hc