New ask Hacker News story: Ask HN: ARM workstations for homelab CPU based AI Inference testing?

Ask HN: ARM workstations for homelab CPU based AI Inference testing?
2 by dylanetaft | 2 comments on Hacker News.
I am eyeing @AmpereComputing's Ampere Altra. There's a docker image for optimized PyTorch https://ift.tt/SqWyLDC I am pretty sure it is just OpenBlas compiled with special options + PyTorch using that, but who knows. There doesn't seem to be a ton of affordable options. I see the Altra has ASIMD\NEON but not SVE. If I have a budget of ~3000, is one of their systems probably the best choice? There are several standard formfactor motherboards and even combo kits available at normal outlets. I'm not _MARRIED_ to ARM, but ~60w idle, 200w tdp when it's busy - that can be cooled quietly, and performance looks decent.

Comments

Popular posts from this blog

How can Utilize Call Center Outsourcing for Increase your Business Income well?

New ask Hacker News story: Is someone trying to steal credit for inventing the eTicket?