Skip to content

Host-weight prefetch in the scheduler, faster MoE cache prompt and decode, cache seeding under llama-server - #268

Open
danielhanchen wants to merge 88 commits into
base/upstream-cbe28bb3cfrom
moe-offload-prefetch-cache
Open

Host-weight prefetch in the scheduler, faster MoE cache prompt and decode, cache seeding under llama-server#268
danielhanchen wants to merge 88 commits into
base/upstream-cbe28bb3cfrom
moe-offload-prefetch-cache

tests : link the MoE cache tests against llama only

fd7489e
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs