fabolous005/prain
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 1e3a412563 prain: serve with the string chat template content format master fabolous005 2026-11-04 21:50:46 +01:00
  • a67238f66b prain: personas for the chats; README on quantization levels and personas fabolous005 2026-10-29 23:55:36 +01:00
  • a14e3ade0b prain: web UI with a model monitor fabolous005 2026-10-29 14:20:23 +01:00
  • d1ccc5c294 prain: int4 MoE on RDNA2: GEMV-style decode kernel, fp16 rounds, staged expert copies fabolous005 2026-10-29 14:20:23 +01:00
  • 0f3911afb5 prain: span profiler with flame graphs (PRAIN_PROFILE: engine step, every model module, MoE route/plan/copy/round, call counts; python -m prain.profile); prain web --host fabolous005 2026-10-29 08:39:28 +01:00
  • bb42ea3802 prain: README: expert formats, dense-layer fix, min_weight trade-off, 16k agent setup; example config int4, 16k, tools fabolous005 2026-10-28 22:07:25 +01:00
  • 216cd76a77 prain: Triton GEMV for decode GEMMs on gfx10 (rocBLAS ran bf16 GEMVs at 6% of the bandwidth; agent setup 1.58 -> 2.77 tok/s, same answers) fabolous005 2026-10-28 22:07:25 +01:00
  • 708e152549 prain: int8/int4 experts quantized once on the GPU, rounds through vLLM's expert_map, [miss] min_weight skips experts not in VRAM; scripts: tune_moe, quality (answer NLL on identical token ids), bench --context, measure --quant fabolous005 2026-10-28 22:07:25 +01:00
  • 3ae9bd73a7 prain: README; example config with the measured eGPU defaults (miss auto <=2 tokens, 16 CPU threads, no pinned RAM tier) fabolous005 2026-10-28 01:10:17 +01:00
  • 16c3bf81f7 prain: pin the RAM tier per expert (big pinned blocks take minutes on eGPU hosts); sim charges disk reads on the CPU path; measure CPU cost on cold experts fabolous005 2026-10-28 00:33:34 +01:00
  • 01dce2ffc6 prain: tune searches the auto CPU threshold and writes --cpu-threads; vLLM patch: fit Gemma 4 in 8 GB (tied lm_head without a copy, rope caches sized by max_model_len) fabolous005 2026-10-27 23:52:53 +01:00
  • d69fd34492 prain: fix vLLM on ROCm (no CUDA capability families), smoke test and benchmark scripts fabolous005 2026-10-27 16:44:40 +01:00
  • 7be39c9287 prain: build torchvision too (transformers Gemma 4 processor imports it) fabolous005 2026-10-27 16:04:22 +01:00
  • 341ad52baa prain: warm-up and draft measurement scripts fabolous005 2026-10-27 15:56:48 +01:00
  • 94cc4c23bb prain: cost model measurement fabolous005 2026-10-27 15:56:48 +01:00
  • 036f4611cc prain: vLLM tiered experts layer and plugin fabolous005 2026-10-27 15:55:57 +01:00
  • 7bab776a16 prain: tiered forward with CPU path fabolous005 2026-10-27 15:54:11 +01:00
  • 366e569541 prain: build scripts (ROCm SDK venv, PyTorch and vLLM from source with clang) fabolous005 2026-10-27 15:42:17 +01:00
  • a49ff1f3b2 prain: RX 6600 XT example config fabolous005 2026-10-27 13:52:26 +01:00
  • 1cf3c18a82 prain: draft rows from the batch layout fabolous005 2026-10-27 13:52:25 +01:00
  • cf9a347792 prain: web UI with sessions and drafts fabolous005 2026-10-27 13:52:25 +01:00
  • 0531aaed44 prain: terminal UI fabolous005 2026-10-27 13:52:16 +01:00
  • 041398d849 prain: chat client with cancellable warm-up fabolous005 2026-10-27 13:52:14 +01:00
  • 7a78cdee79 prain: command line fabolous005 2026-10-27 13:52:10 +01:00
  • 168251e81c prain: traces, simulator and tuner fabolous005 2026-10-27 13:52:09 +01:00
  • 582601101b prain: slot cache with eviction, scan, prefetch and CPU policies fabolous005 2026-10-27 13:52:07 +01:00
  • 8b34961308 prain: stats planner and TOML writer fabolous005 2026-10-27 13:52:06 +01:00
  • a5f011dcb8 prain: placement and option tables fabolous005 2026-10-27 13:52:06 +01:00
  • db1736c42d prain: expert id ranges fabolous005 2026-10-27 13:52:05 +01:00
  • c8d5706956 prain: skeleton fabolous005 2026-10-27 13:52:04 +01:00