Search Model Serving

Found 99 bookmarks

Newest

Alishahryar1/free-claude-code: Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

Use Claude Code, Codex, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (...

AI Serving Engines #Type: Open-Source #Source Code: GitHub

·github.com·today at 9:04 AM

microsoft/BitNet: Official inference framework for 1-bit LLMs

Official inference framework for 1-bit LLMs. Contribute to microsoft/BitNet development by creating an account on GitHub.

#Type: Open-Source #Source Code: GitHub

·github.com·today at 8:56 AM

microsoft/BitNet: Official inference framework for 1-bit LLMs

lkarlslund/laya.cpp: C++ inference for Laya typed decisions - supports CUDA, Vulkan, Core ML, CPU

C++ inference for Laya typed decisions - supports CUDA, Vulkan, Core ML, CPU - lkarlslund/laya.cpp

AI Serving Engines #Type: Open-Source #Source Code: GitHub

·github.com·Oct 10, 2026

lkarlslund/laya.cpp: C++ inference for Laya typed decisions - supports CUDA, Vulkan, Core ML, CPU

Niko1221/Strata · GitHub

Qwen3.8-Flash-Next (125B MoE) on a 12-24 GB NVIDIA GPU + 64 GB RAM: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input

AI Serving Engines #Type: Open-Source #Source Code: GitHub

·github.com·Oct 5, 2026

Niko1221/Strata · GitHub

Ollaya · Run decision models locally

Source-code: https://github.com/ollaya-dev/ollaya

AI Serving Engines #Type: Open-Source #Source Code: GitHub #❤️

·ollaya.dev·Sep 29, 2026

Ollaya · Run decision models locally

Fast Inference | Fast hosted models, one API

Run high-throughput open models through one OpenAI-compatible API, with transparent pricing, project-scoped keys, and built-in usage analytics.

AI Serving Engines

·fast.inference.net·Sep 22, 2026

Fast Inference | Fast hosted models, one API

DevPass by LLM Gateway - All-Access Dev Plans for AI Coding

One subscription, every coding model. Fixed-price dev plans for Claude Code, Cursor, and 200+ models.

AI Serving Engines

·devpass.llmgateway.io·Sep 21, 2026

DevPass by LLM Gateway - All-Access Dev Plans for AI Coding

Desert Ant Labs: On-device AI models and SDKs

Desert Ant Labs is a frontier AI lab for on-device models. Specialized models that run on Apple, Android, and the web, dropped into any product in a few lines of native code.

AI Local On-Device #Software: AI

·desertant.com·Sep 12, 2026

Desert Ant Labs: On-device AI models and SDKs

Experiential Labs · The open source AI gateway

Source-code: https://github.com/experientiallabs/experiential

AI Serving Engines #Type: Open-Source #Source Code: GitHub

·experientiallabs.ai·Aug 31, 2026

Experiential Labs · The open source AI gateway

oMLX — LLM inference, optimized for your Mac

Source-code: https://github.com/jundot/omlx

AI Local On-Device #Type: Open-Source #Source Code: GitHub #Web server: Self-Host

·omlx.ai·Aug 19, 2026

oMLX — LLM inference, optimized for your Mac

Waku — one native app for all your coding agents

Source-code: https://github.com/egoist/waku

AI Local On-Device #Type: Open-Source #Source Code: GitHub #❤️

·waku.sh·Aug 19, 2026

Waku — one native app for all your coding agents

mlx-optiq · Run LLMs locally on your Mac

Open-source: https://pypi.org/project/mlx-optiq/

AI Local On-Device #Type: Open-Source

·mlx-optiq.com·Aug 8, 2026

mlx-optiq · Run LLMs locally on your Mac

Modal: High-performance AI infrastructure

Bring your own code, and run CPU, GPU, and data-intensive compute at scale. The serverless platform for AI and data teams.

AI Serving Engines

·modal.com·Aug 4, 2026

Modal: High-performance AI infrastructure

E2B | The Enterprise AI Agent Cloud

E2B Gives AI Agents Secure Computers with Real-World Tools. E2B is Used by 94% of Fortune 100 Companies for Frontier Agentic Workflows.

AI Serving Engines

·e2b.dev·Aug 4, 2026

E2B | The Enterprise AI Agent Cloud

Daytona - Secure Infrastructure for Running AI-Generated Code

Deploy Al code with confidence using Daytona's lightning-fast infrastructure. 90ms environment creation, stateful operations, and enterprise-grade security.

AI Serving Engines

·daytona.io·Aug 4, 2026

Daytona - Secure Infrastructure for Running AI-Generated Code

drumih/turbo-fieldfare · GitHub

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook - drumih/turbo-fieldfare

AI Serving Engines #Software: AI #Software: LLM #Source Code: GitHub #Type: Open-Source

·github.com·Jul 30, 2026

drumih/turbo-fieldfare · GitHub

OmniRoute — Free AI Gateway for Multi-Provider LLMs

Source-code: https://github.com/diegosouzapw/OmniRoute

AI Serving Engines #Software: AI #Type: Open-Source

·omniroute.online·Jul 20, 2026

OmniRoute — Free AI Gateway for Multi-Provider LLMs

Tokenomy - Predict. Optimize. Ship AI with confidence.

Tokenomy forecasts token count, dollars, latency & energy before an API call, so your team ships without surprise bills.

AI Serving Engines #Software: AI

·tokenomy.ai·Jul 11, 2026

Tokenomy - Predict. Optimize. Ship AI with confidence.

9Router - Free AI Router | Smart Fallback for Claude, Codex & More

Source-code: https://github.com/decolua/9router

AI Serving Engines #Type: Open-Source #Source Code: GitHub

·9router.com·Jul 10, 2026

9Router - Free AI Router | Smart Fallback for Claude, Codex & More

lyogavin/airllm: AirLLM 70B inference with single 4GB GPU · GitHub

AirLLM 70B inference with single 4GB GPU

AI Serving Engines #Software: LLM #Source Code: GitHub #Type: Open-Source #Software: AI

·github.com·Jul 2, 2026

lyogavin/airllm: AirLLM 70B inference with single 4GB GPU · GitHub

ayushh0110/ScreenMind: AI-powered screen memory — captures, analyzes, and lets you search/chat your screen history. Powered by Gemma 4 . 100% local, 100% private.

AI-powered screen memory — captures, analyzes, and lets you search/chat your screen history. Powered by Gemma 4 . 100% local, 100% private. - ayushh0110/ScreenMind

AI Local On-Device #Software: AI #Type: Open-Source #Source Code: GitHub

·github.com·Jun 22, 2026

ayushh0110/ScreenMind: AI-powered screen memory — captures, analyzes, and lets you search/chat your screen history. Powered by Gemma 4 . 100% local, 100% private.

Unsloth - Train and Run Models Locally

Source-code: https://github.com/unslothai/unsloth

AI Serving Engines #Software: AI #Web server: Self-Host #Type: Open-Source

·unsloth.ai·Jun 19, 2026

Unsloth - Train and Run Models Locally

Local Waifu | A self-learning AI companion for macOS

A self-learning AI companion that lives on your Mac. She remembers you, listens, learns who you are. No cloud, no subscription. $25 once, 7 days free.

AI Local On-Device #OS compatibility: macOS #Software: AI #Device: Desktop

·localwaifu.com·Jun 10, 2026

Local Waifu | A self-learning AI companion for macOS

MLX — MLX 0.31.2 documentation

Source-code: https://github.com/ml-explore/mlx

AI Serving Engines #Type: Open-Source #Software: Library #Software: AI

·ml-explore.github.io·Jun 10, 2026

MLX — MLX 0.31.2 documentation

OpenJarvis

Source-code: https://github.com/open-jarvis/OpenJarvis

AI Local On-Device #Type: Open-Source #Source Code: GitHub #Software: AI #Web server: Self-Host

·openjarvis.stanford.edu·Jun 10, 2026

OpenJarvis

Andyyyy64/whichllm: Find the local LLM that actually runs and performs best on your hardware

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly. - Andyyyy64/whichllm

AI Serving Engines #Software: CLI #Source Code: GitHub #Type: Open-Source

·github.com·Jun 9, 2026

Andyyyy64/whichllm: Find the local LLM that actually runs and performs best on your hardware