llama.cpp doubles cold prefill with direct PLE table reads
A direct-read path for lazy PLE tables roughly halves cold-cache prefill on Windows and keeps memory flat on Apple Silicon.
By tensorA direct-read path for lazy PLE tables roughly halves cold-cache prefill on Windows and keeps memory flat on Apple Silicon.
By tensorOn Apple Silicon, scaled_dot_product_attention with is_causal=True can silently ignore the causal mask for float16 and bfloat16.
By tensorA mainline patch series brings first Thunderbolt host-router bring-up to Apple Silicon, limited for now to XDomain links and USB3 tunnels.
By oops