psmarter / mini-infer Star 56 Code Issues Pull requests A high-performance LLM inference engine with PagedAttention | 基于PagedAttention的高性能大模型推理引擎 python machine-learning ai deep-learning gpu cuda inference pytorch transformer triton language-model llm pagedattention Updated Dec 31, 2025 Python
gty111 / gLLM Star 55 Code Issues Pull requests Discussions gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling pipeline-parallelism tensor-parallelism llm-serving llm-inference pagedattention continuous-batching qwen3 token-throttling chunked-prefill Updated Jan 12, 2026 Python
jmaczan / tiny-vllm Star 8 Code Issues Pull requests High performance LLM inference engine, a younger sibling of vLLM ai cpp hpc cuda llm vllm llm-inference pagedattention tiny-vllm Updated Mar 6, 2026 C++
Rianbajukendari / mini-infer Star 0 Code Issues Pull requests 🚀 Accelerate LLM inference with Mini-Infer, a high-performance engine designed for efficiency and power in AI model deployment. python machine-learning ai deep-learning gpu cuda inference pytorch transformer triton language-model llm pagedattention Updated Mar 6, 2026 Python