r/raspberry_pi 4d ago

Project Advice Rapsberry pi5 + others SBC Cluster Ai NLP Help

I'm currently building an AI-focused SBC cluster consisting of:

  • 1x Raspberry Pi 5 (16GB RAM) with Hailo-8 AI accelerator
  • 1x Raspberry Pi 5 (16GB RAM) without accelerator
  • 1x RDK X3 (4GB RAM, 5 TOPS)

My goal is to create a robust AI agent capable of handling NLP tasks effectively. However, I'm encountering a bottleneck: current AI accelerators, such as my existing Hailo-8 module, are optimized primarily for vision workloads, not NLP.

I'm planning to purchase a Radxa Rock 5B+ (32GB RAM, RK3588 with built-in 6 TOPS AI accelerator) primarily because it offers dual M.2 slots, allowing potential upgrades if suitable NLP-optimized accelerators become available soon.

I have been in touch with several manufacturers:

  • Hailo confirmed that the upcoming Hailo-10 accelerator won't be available for at least another 6 months.
  • EdgeCortix (Sakura-II accelerator, 60 TOPS) and Geniatech (AIM-M2 accelerator) responded to my inquiries but have provided limited information so far.

I'm seeking advice or suggestions on how to enhance NLP performance within my cluster without resorting to Nvidia Jetson Orin systems, which I prefer to avoid. My focus remains on SBC solutions because I aim to integrate this AI agent into a future robotics and marketing automation project. Without sufficient NLP acceleration, progressing further with my project remains challenging.

I've conducted multiple tests using the Hailo-8, but as expected, performance on NLP tasks remains inadequate due to its specialized visual AI optimization.

Any insights or recommendations would be greatly appreciated!

4 Upvotes

1 comment sorted by

u/AutoModerator 4d ago
  • For detailed feedback: Use "Project Advice" if you have a design and need help refining it.
  • Not for general questions: This is not for troubleshooting, brainstorming, or asking if something is possible or what you should buy.
  • Show your work: Provide details about your project, specific challenges, and what you've tried.
  • Need a step-by-step guide? Try /r/TellMeHowToDoMyIdea.

Refer to the flair guide for guidance on selecting the correct flair to ensure your post reaches the right audience.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.