把微软针对cpu优化的ai大模型跑起来了，纯cpu跑 27.09 tokens/s
[*]速度是真快，英语效果一般，中文效果堪忧！:lol[*]
[*]llama_perf_sampler_print: sampling time = 108.23 ms /1040 runs ( 0.10 ms per token,9608.72 tokens per second)
[*]llama_perf_context_print: load time = 582.45 ms
[*]llama_perf_context_print: prompt eval time = 327.41 ms / 10 tokens ( 32.74 ms per token, 30.54 tokens per second)
[*]llama_perf_context_print: eval time = 37991.07 ms /1029 runs ( 36.92 ms per token, 27.09 tokens per second)
[*]llama_perf_context_print: total time = 38726.16 ms /1039 tokens
[*]复制代码
叫bitnet
页:
[1]