minus-squareCaptnBook@feddit.orgtoSelfhosted@lemmy.world•Faster Ollama alternativelinkfedilinkEnglisharrow-up2·13 hours agoVllm unfortunately doesn’t support switching the model without a restart. linkfedilink
minus-squareCaptnBook@feddit.orgtoSelfhosted@lemmy.world•Faster Ollama alternativelinkfedilinkEnglisharrow-up1·edit-213 hours agoIt’s not, by far. But vllm or SGLang don’t support switching the model… such a shame. linkfedilink
Vllm unfortunately doesn’t support switching the model without a restart.