 fabolous005
|
bb42ea3802
|
prain: README: expert formats, dense-layer fix, min_weight trade-off, 16k agent setup; example config int4, 16k, tools
|
2026-10-28 22:07:25 +01:00 |
|
|
fabolous005
|
216cd76a77
|
prain: Triton GEMV for decode GEMMs on gfx10 (rocBLAS ran bf16 GEMVs at 6% of the bandwidth; agent setup 1.58 -> 2.77 tok/s, same answers)
|
2026-10-28 22:07:25 +01:00 |
|
|
fabolous005
|
708e152549
|
prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant
|
2026-10-28 22:07:25 +01:00 |
|
|
fabolous005
|
3ae9bd73a7
|
prain: README; example config with the measured eGPU defaults (miss auto <=2 tokens, 16 CPU threads, no pinned RAM tier)
|
2026-10-28 01:10:17 +01:00 |
|
|
fabolous005
|
16c3bf81f7
|
prain: pin the RAM tier per expert (big pinned blocks take minutes on eGPU hosts); sim charges disk reads on the CPU path; measure CPU cost on cold experts
|
2026-10-28 00:33:34 +01:00 |
|
|
fabolous005
|
01dce2ffc6
|
prain: tune searches the auto CPU threshold and writes --cpu-threads; vLLM patch: fit Gemma 4 in 8 GB (tied lm_head without a copy, rope caches sized by max_model_len)
|
2026-10-27 23:52:53 +01:00 |
|
|
fabolous005
|
d69fd34492
|
prain: fix vLLM on ROCm (no CUDA capability families), smoke test and benchmark scripts
|
2026-10-27 16:44:40 +01:00 |
|
|
fabolous005
|
7be39c9287
|
prain: build torchvision too (transformers Gemma 4 processor imports it)
|
2026-10-27 16:04:22 +01:00 |
|
|
fabolous005
|
341ad52baa
|
prain: warm-up and draft measurement scripts
|
2026-10-27 15:56:48 +01:00 |
|
|
fabolous005
|
94cc4c23bb
|
prain: cost model measurement
|
2026-10-27 15:56:48 +01:00 |
|
|
fabolous005
|
036f4611cc
|
prain: vLLM tiered experts layer and plugin
|
2026-10-27 15:55:57 +01:00 |
|
|
fabolous005
|
7bab776a16
|
prain: tiered forward with CPU path
|
2026-10-27 15:54:11 +01:00 |
|
|
fabolous005
|
366e569541
|
prain: build scripts (ROCm SDK venv, PyTorch and vLLM from source with clang)
|
2026-10-27 15:42:17 +01:00 |
|
|
fabolous005
|
a49ff1f3b2
|
prain: RX 6600 XT example config
|
2026-10-27 13:52:26 +01:00 |
|
|
fabolous005
|
1cf3c18a82
|
prain: draft rows from the batch layout
|
2026-10-27 13:52:25 +01:00 |
|
|
fabolous005
|
cf9a347792
|
prain: web UI with sessions and drafts
|
2026-10-27 13:52:25 +01:00 |
|
|
fabolous005
|
0531aaed44
|
prain: terminal UI
|
2026-10-27 13:52:16 +01:00 |
|
|
fabolous005
|
041398d849
|
prain: chat client with cancellable warm-up
|
2026-10-27 13:52:14 +01:00 |
|
|
fabolous005
|
7a78cdee79
|
prain: command line
|
2026-10-27 13:52:10 +01:00 |
|
|
fabolous005
|
168251e81c
|
prain: traces, simulator and tuner
|
2026-10-27 13:52:09 +01:00 |
|
|
fabolous005
|
582601101b
|
prain: slot cache with eviction, scan, prefetch and CPU policies
|
2026-10-27 13:52:07 +01:00 |
|
|
fabolous005
|
8b34961308
|
prain: stats planner and TOML writer
|
2026-10-27 13:52:06 +01:00 |
|
|
fabolous005
|
a5f011dcb8
|
prain: placement and option tables
|
2026-10-27 13:52:06 +01:00 |
|
|
fabolous005
|
db1736c42d
|
prain: expert id ranges
|
2026-10-27 13:52:05 +01:00 |
|
|
fabolous005
|
c8d5706956
|
prain: skeleton
|
2026-10-27 13:52:04 +01:00 |
|