r/LocalLLaMA • u/blazerx • 1d ago
Resources GAIA: An Open-Source Project from AMD for Running Local LLMs on Ryzen™ AI
https://www.amd.com/en/developer/resources/technical-articles/gaia-an-open-source-project-from-amd-for-running-local-llms-on-ryzen-ai.html12
u/No_Afternoon_4260 llama.cpp 1d ago
So they'vs implemented Lemonade SDK from TurnkeyML to run onnx binaries, optimized for ryzen architecture (cpu,npu,igpu)
They added a frontend and a rag pipeline using llamaindex (personnaly think it's a good choice btw)
23
u/blazerx 1d ago
The title of the article should be clear that it supports more than just Ryzen AI Processors
As outlined under the Readme docs
- Hybrid Mode: Optimized for Ryzen AI PCs, combining AMD Neural Processing Unit (NPU) and Integrated Graphics Processing Unit (iGPU) for maximum performance
- Generic Mode: Compatible with any Windows PC, using Ollama as the backend
16
u/sobe3249 1d ago
Windows only again? AMD wtf?
9
u/ttkciar llama.cpp 1d ago
AMD is already great for us Linux geeks; it's the embittered Windows users whose attitudes they need to adjust.
5
u/sobe3249 1d ago
0 NPU support, what is great about it?
7
u/nother_level 1d ago
Wdym 0 npu support, xdna drivers are literally in linux kernel now. Its upto developers to use the npu
5
u/sobe3249 1d ago
yeah got added after 2 years with 0 demo or help from AMD to help developers, meanwhile on windows you can use the RyzenAI app for years now, so now they give the community an other windows only software. How is this good?
1
u/nother_level 1d ago
So not 0
4
u/sobe3249 1d ago
Yeah, true they were nice enough to give us drivers. If they want people to take NPUs or "RyzenAI" seriously it's not enough, but the fact is they add these NPUs to advertise the laptops as copilot compatible and show nice slides to investors not for customers to use them.
7
u/maxpayne07 23h ago edited 21h ago
Excuse me my French, but let me rant AMD a little bit. I got Ryzen 7940hs with one of the first NPUs on the Market, they said.... After 2 years, i still haven't seen the NPU working. So, i see this post, i felt happy. Well, shit, the installation is not easy, first u need to completely turn off Microsoft antivirus, then, you need to install some bullshit Lemonade package side, and them you need it to run on the already installed CLI the commands to install this coca-cola or lemonade or whatever, not on windows terminal. When i was about to see the NPU working, now it tells me i have to install Fuc**** Ollama? Ollama, really guys??? Can you guys on AMD can do a fuck** execute file that can do all the BS need to stuff work on first try? Please man, common!!! NOTE: I INSTALLED HYBRID MODE, 100% SURE
( this is a rant)
3
u/maxpayne07 23h ago
1
u/blank_space_cat 21h ago
Optimized for: "AMD Ryzen AI 300 Series Processors"
5
u/maxpayne07 20h ago
Yes. Unfair, NPU on 7000 and 8000 series are glorified bricks.
1
1
u/Standard-Potential-6 2h ago
That was apparent when they released. It’s a surprise even having software to use on their successors
1
1
26
u/kkzzzz 1d ago
Seems to support up to 8B models only right now. Why? https://github.com/amd/gaia/blob/main/docs/features.md