BentoML

BentoML/Bento is an inference platform (from Modular) for deploying AI/ML models to production — self-hosting on any cloud, on-premises, or Kubernetes, with a free open-source core and a paid managed cloud layer for scaling. Who it's for: ML engineering teams who need to deploy and serve models — open-source (Llama, DeepSeek, Qwen) or custom — in production, with real control over infrastructure and cost rather than being locked into one hosted inference API. Strengths: the open-source BentoML framework being genuinely free means a team can self-host at zero licensing cost, with the managed Bento Inference Platform available when a team wants that operational burden handled for them. Intelligent scaling (auto-scaling, cold-start acceleration, scale-to-zero) directly addresses the real cost/latency tradeoff of running model inference at variable load. Weaknesses and limitations: pricing for the managed cloud platform (and specific GPU options like H100/B200) isn't published — real cost requires booking a demo, and self-hosting the open-source version still requires real infrastructure expertise. Real-world scenario: an ML team needs to deploy a custom or open-source model to production with control over cost and infrastructure, rather than depending entirely on a third-party hosted inference API.