inference AI Tools

Fast inference API for 100+ open-source LLMs at competitive prices.

Official inference framework for 1-bit LLMs, by Microsoft.

a high-speed inference engine for deploying LLMs locally

LPU inference engine delivering the fastest AI model response times.

Inference of Meta's LLaMA model (and others) in pure C/C++. #opensource