Please don't use this tool. - it is a massive intellectual property risk

[ArgonianVoter]

make it stooooooooooooooooooop

 

[Battlefield42]

Ver archivo adjunto 8627140
make it stooooooooooooooooooop

*nukes your bookmarks*

 

[Mad as a Swan]

If at any point there was a GitHub leak, your code is public.

 

[Lian Xing]

So there is the AMD Radeon Pro 9700 which has 32GB VRAM and is less than half the cost of the 5090. But also less than half the speed at these tasks. It's probably enough for my purposes, but I'm trying to find good benchmarks.

Is this a use case for CrossFire or SLI?

 

[Overly Serious]

Is this a use case for CrossFire or SLI?

I don't think that's a thing anymore on the current gen AMD cards. I'm reasonably sure that the software used for LLMs just manages the cards directly automatically, and with modern PCI-E speeds (i.e. v4 and v5), special cross-fire support isn't needed. You can certainly run an LLM across multiple cards with llama.cpp. How performant that is, depends largely on the suitability of the model itself. Like I think diffusion models in image generation don't run across cards well, but some LLMs do it well. It's one of those cases you really have to have a good idea before you buy of what you actually want to achieve.

I weighed up what I needed, as a follow-up btw, and whilst paired Radeon Pro 9700s are a very cost effective solution, I felt they didn't quite fit my use case and have held off that for the time being. I am looking at Nvidia's RTX Pro Blackwell line. The jump in price over the AMD solution is BIG, though.

 

[Staffy]

Jeets will surely use something like this for any projects they unfortunately are a part of, not thinking about the consequences.