prain: web UI with a model monitor

A clear chat page (markdown, code blocks with copy, stop, new chat, kept in the browser, drafts shown as such) and a monitor sidebar, a sheet on phones: tokens/s, a live map of the 30 x 128 experts (in VRAM, used now, copied in), VRAM hit rate, copies, context, prompt cache, GPU VRAM/load/temperature/power/fan. GET /status merges vLLM /metrics, amdgpu sysfs and the stats file, which now lists the experts in VRAM. /warm answers 202 when a warm-up starts. README: the int4 kernels, their A/B, 2048-token chunks as a trade, prompt logprobs limit.
2026-10-29 14:20:23 +01:00
parent d1ccc5c294
commit a14e3ade0b
11 changed files with 977 additions and 133 deletions
@@ -119,3 +119,7 @@ class Prefetch(TmpCase):
            stats = json.load(f)["layers"]["7"]
        self.assertEqual((stats["uses"], stats["prefetch_wasted"], stats["misses"]),
                         ([1, 1, 1, 0], 1, 2))
+        self.assertEqual(stats["vram"], [1, 2])  # the experts in VRAM now, for the web monitor
+        write_stats(path, {7: SlotCache([5, 3], 8, 2, Policy(eviction="lfu"))})
+        with open(path) as f:
+            self.assertEqual(json.load(f)["layers"]["7"]["vram"], [3, 5])  # pinned ones too