Is it possible to run DeepSeek-R1-Distill-Llama-70B or a 4-bit quantified version on consumer grade computers? This is of great significance

[wusu2016]

1.例如显存在 16-22GB之间，兼容 4060ti,4070,4080,4090，此类用户不需要更换显卡，节省费用，避免去为了SLI, 去弄显卡交火，增加成本
2.70B比 32B， 14B 聪明，用于写代码，绘图够用，
3.deepseek已是国内较知名的模型， 流量大
4. 最好可以提升tokes/s,节省使用时间
此项目必火

[ejrydhfs]

Not right now as this project currently is optimized for a few transformer ai model architectures and deepseek R1 does not use one of them. I believe it would be best to work on making it work in a fork since this project has not been updated in a long time