New ask Hacker News story: Ask HN: 2x Arc A770 or 1x Radeon 7900 XTX for llama.cpp
Ask HN: 2x Arc A770 or 1x Radeon 7900 XTX for llama.cpp
2 by danielEM | 1 comments on Hacker News.
Can't find "apple to apple" comparison on performance on QWQ 32b (4bit), can anyone help me with decision on which solution to pick? From what I dig so far it looks like dual Arc A770 is supported by llama.cpp. And saw some reports that llama.cpp on top of IPEX-LLM is fastest way for inference on intel card. On the other end there is more expensive 7900 XTX on which AMD claims (Jan '25) that inference is faster than on 4090. So - what is the state of the art as of today, how does one compare to another (apple to apple)? What is tokens/s diff?
2 by danielEM | 1 comments on Hacker News.
Can't find "apple to apple" comparison on performance on QWQ 32b (4bit), can anyone help me with decision on which solution to pick? From what I dig so far it looks like dual Arc A770 is supported by llama.cpp. And saw some reports that llama.cpp on top of IPEX-LLM is fastest way for inference on intel card. On the other end there is more expensive 7900 XTX on which AMD claims (Jan '25) that inference is faster than on 4090. So - what is the state of the art as of today, how does one compare to another (apple to apple)? What is tokens/s diff?
Comments
Post a Comment