SYCL_CACHE_PERSISTENT=1 causes crashes in llama.cpp; not recommended
[major] config
Fields
id
gotchas/xpu-sycl-cache-persistent-crashClass
config
Severity
major
Concepts
sycloneapillama.cpp
Symptom
Setting SYCL_CACHE_PERSISTENT=1 in the environment causes crashes when running llama.cpp with the SYCL backend. The crash occurs because the SYCL runtime caches and reuses JIT-compiled binaries, and when the binary file changes (e.g., due to a code update), the new and old codes mix, leading to a crash.
Root cause
The SYCL runtime's persistent cache mechanism is not safe for production use when the underlying code changes frequently. The cache is stored in ~/.cache/libsycl_cache/ and persists across runs.
Workaround
Do not set SYCL_CACHE_PERSISTENT=1. If you have already set it and experience crashes, remove the local cache: rm -rf ~/.cache/libsycl_cache/ and unset SYCL_CACHE_PERSISTENT. Note that some AI assistants recommend this setting to 'speed up SYCL backend' - this is incorrect advice; it only speeds up startup by skipping JIT compilation, not running speed.
notes
This is a known issue documented in the llama.cpp SYCL backend docs. The crash is caused by mixing old and new JIT-compiled binaries. The setting is not recommended by the llama.cpp SYCL backend team.
Notes
This is a known issue documented in the llama.cpp SYCL backend docs. The crash is caused by mixing old and new JIT-compiled binaries. The setting is not recommended by the llama.cpp SYCL backend team.