Respuestas al mensaje #685

🇵🇦 Nuestro primer dominio localizado está en español en kiwifarms.pa. Our first localized domain is on Spanish on kiwifarms.pa.

How are you mixing RAM and VRAM? Is there a setting I'm missing here? I can't run a big 70B model unless I spin up dual 3090s so it all fits in VRAM.

What gets me the most is the speed at which things are going. A year ago running a model on the level of llama 2 70b on home hardware would've...