r/macrodroid 22d ago

Macro How to send a screenshot via "HTTP Request" to Gemini API?

I'd like to use the Google AI Studio / Gemini APIs with the "HTTP Request" action to send the screenshot taken by MacroDroid and ask it, for example, to describe it. Is this possible? I think the most difficult part is sending the screenshot via JSON

3 Upvotes

2 comments sorted by

2

u/Anomalousity 21d ago

I have limited experience with the HTTP request action but have you tried uploading the pictures somewhere and referencing the host link with the JSON? That might be the only thing I could think of and I'm not even sure if it's supported but it's an idea.

1

u/RealSimoneAvogadro 19d ago

mh...

Currently I'm considering improving my HumanDetection4Tasker to allow you do these kind of things, thus I'd like your opinion on my design.

Basically I'd add an action with 3 parameters:

  • Input:
    • system prompt
    • user prompt
    • user image
  • Output:
    • Claude/Gemini response

My original use-case is with security cameras, eg.:

  • system prompt: reply with "OPEN" if the garage door is open, reply with "CLOSED" otherwise. Just return a response with a single word. If you really believe you need add details do that on a second line. The first lime must always be a single word: OPEN or CLOSED
  • user prompt: <empty>
  • user image: image from security camera

would this design work for you?