Join Vultr Community
Points, levels & badges
Enter challenges & submit builds
Register to events & meetups
Get your work featured
Serverless Interference - Possibility to add cache system?
Joesph Koen
NoviceMany places that host their own models have the ability of caching.
Any possible plans to add caching in the future? I like the models offered, but the pricing is hard to justify compared to original API alternatives. I would like to use vultr for everything, been my main vps provider for years now, and the vector store feature is quite handy, but long term price is not justified for use.
MA
1 comment1 Comment
1
Commented: 5 months ago
I agree that a caching layer could significantly improve the value proposition of serverless inference platforms. For applications with repeated prompts, embeddings, or common requests, caching can reduce both latency and operational costs while improving overall efficiency.
The vector store feature is already a useful addition, and combining it with intelligent caching could make the platform more competitive for long-term production workloads. Cost optimization is becoming increasingly important as AI applications scale.
I also follow discussions around cloud infrastructure, storage optimization, and digital accessibility on terabox mod apk.
It would be interesting to hear whether the Vultr team has any plans for response caching, prompt caching, or semantic caching in future updates.
1