https://friendli.ai/blog/friendli-tcache-flexible-multimodal-prefix-caching
Friendli TCache: Flexible Multimodal Prefix Caching
Friendli TCache expands prefix caching beyond text, enabling support for multimodal inputs like image and video embeddings. This unlocks faster inference and...
friendlitcacheflexiblemultimodalprefix
https://friendli.ai/blog/friendli-container-llm-on-premise
Friendli Container Part 1: Efficiently Serving LLMs On-Premise
Friendli Dedicated Endpoints offer a simple, secure, reliable, and cost-efficient solution, some industry sectors, such as finance, may prefer on-premise...
serving llmsfriendlicontainerpartefficiently