fabolous005 1e3a412563 prain: serve with the string chat template content format

2026-11-04 21:50:46 +01:00

..

__init__.py

prain: skeleton

2026-10-27 13:52:04 +01:00

helpers.py

prain: web UI with a model monitor

2026-10-29 14:20:23 +01:00

test_chat.py

prain: chat client with cancellable warm-up

2026-10-27 13:52:14 +01:00

test_cli.py

prain: serve with the string chat template content format

2026-11-04 21:50:46 +01:00

test_config.py

prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant

2026-10-28 22:07:25 +01:00

test_drafts.py

prain: draft rows from the batch layout

2026-10-27 13:52:25 +01:00

test_gemv.py

prain: Triton GEMV for decode GEMMs on gfx10 (rocBLAS ran bf16 GEMVs at 6% of the bandwidth; agent setup 1.58 -> 2.77 tok/s, same answers)

2026-10-28 22:07:25 +01:00

test_kernel_match.py

prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant

2026-10-28 22:07:25 +01:00

test_kernels.py

prain: int4 MoE on RDNA2: GEMV-style decode kernel, fp16 rounds, staged expert copies

2026-10-29 14:20:23 +01:00

test_plugin.py

prain: vLLM tiered experts layer and plugin

2026-10-27 15:55:57 +01:00

test_policy.py

prain: web UI with a model monitor

2026-10-29 14:20:23 +01:00

test_profile.py

prain: span profiler with flame graphs (PRAIN_PROFILE: engine step, every model module, MoE route/plan/copy/round, call counts; python -m prain.profile); prain web --host

2026-10-29 08:39:28 +01:00

test_quant.py

prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant

2026-10-28 22:07:25 +01:00

test_sim.py

prain: pin the RAM tier per expert (big pinned blocks take minutes on eGPU hosts); sim charges disk reads on the CPU path; measure CPU cost on cold experts

2026-10-28 00:33:34 +01:00

test_tiered.py

prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant

2026-10-28 22:07:25 +01:00

test_tui.py

prain: personas for the chats; README on quantization levels and personas

2026-10-29 23:55:36 +01:00

test_web_page.py

prain: web UI with a model monitor

2026-10-29 14:20:23 +01:00

test_web.py

prain: personas for the chats; README on quantization levels and personas

2026-10-29 23:55:36 +01:00