BF16 weight prepack needs the cpu support avx512bw, avx512vl and avx512dq - how to set up neural chat to AVX2

[hnguy31-hvevn]

Hi Team,

I have an issue with CPU XEON GOLD , Ubuntu VM.
How do I set up neural chat to AVX2?

BF16 weight prepack needs the cpu support avx512bw, avx512vl and avx512dq

[2024-04-23 00:52:31,123] [ ERROR] - Failed to start server.
[2024-04-23 00:52:31,123] [ ERROR] - BF16 weight prepack needs the cpu support avx512bw, avx512vl and avx512dq, but the desired instruction sets are not available. Please set dtype to torch.float or set weights_prepack to False.

[Spycsh]

Hi @hnguy31-hvevn , yes I think IPEX may optimize too aggressively. There is actually a parameter to turn off the weights_prepack.

I've made a PR here #1526, could you have a look on whether this fix your issue?