LLMLingua Series | Effectively Deliver Information to LLMs via Prompt Compression
Source-code: https://github.com/microsoft/LLMLingua
BentoML: Build, Ship, Scale AI Applications
Source-code: https://github.com/bentoml/OpenLLM
cumulo-autumn/StreamDiffusion · GitHub
StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation - cumulo-autumn/StreamDiffusion: StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation
Amazon Q - AWS
Llama.cpp - Run LLM Inference in C/C++
Source-code: https://github.com/ggml-org/llama.cpp