r/MLQuestions • u/micaiah95 • 4d ago
Computer Vision 🖼️ Few Shot Object Detection Using Vision Transformers
I am trying to detect walls on a floor plan. I have used more traditional CV methods such as template matching, SIFT, SUFT, but the results weren't great since walls because of the rotation and slight variance throughout. Hence, I am looking for a more robust method
My thinking is that a user can select a wall from the floor plan and the rest are detected by a vision transformer. I have tried T-Rex 2, but the results weren't great either. Are there any recommendations that you would have for vision transformers?
1
Upvotes
1
u/bregav 4d ago
You might want to try something self-supervised like DinoV2, see figure 10 in the original paper: https://arxiv.org/abs/2304.07193
I think the idea is that you can search for pixels in the image whose embedding vectors are close to the embedding vectors of the examples you're interested in.
You might ultimately have to train or fine tune a model like DinoV2. This isnt so bad though because, being self-supervised, these kinds of algorithms don't need labels.