Redis Vector Sets and LangCache Speed GenAI Models
Redis has unveiled its latest advancements in data management with the introduction of Vector Sets and the LangCache semantic caching service. These offerings are designed to enhance the development and execution of GenAI (Generative AI) models by delivering speed and efficiency.
Redis, known as Remote Dictionary Server, is a developer and provider of a real-time, in-memory key-value store structure. This structure is versatile, functioning as a cache, database, or message broker, with the capability to write persistent data to attached storage drives. Its support extends across a variety of data types including strings, lists, sorted and unsorted sets, hashes, bitmaps, streams, and vectors. The recent addition of Vector Set data types marks a significant expansion of these capabilities.
The advent of the Vector Set complements Redis’s existing vector similarity search by offering a more granular method of working with vectors. Meanwhile, LangCache empowers developers to integrate Redis’s LLM (Large Language Model) response caching into their applications, effectively reducing the frequency of LLM calls. This cache stores and reuses prompts and responses, leading to lower costs, improved prompt accuracy, and faster responsiveness.
CEO Rowan Trollope elaborated that the “Vector Set is modelled on a sorted set but it’s hyper-dimensional.” While traditional sorted sets have string elements tied to a score, the Vector Set links these elements to a vector.
Salvatore Sanfilippo, the creator of Redis and one of Trollope’s advisors, crafted the Vector Set by devising an algorithm compatible with quanticized vectors. These vectors are significantly smaller than the typical 32-bit vectors, allowing for more storage in RAM and enhancing the efficacy of semantic searches.
The primary objective of Vector Sets is to enable the addition of items and to fetch subsets of these items that most closely resemble a specified vector. As Trollope states, “We can perform natural language searches on this Redis Vector Set.”
Delving deeper, Vector Sets implement:
- Quanticization that enables the use of int8 embeddings, reducing memory usage and cost by up to 75%.
- Enhanced search speeds by 30%, while maintaining 99.99% search accuracy.
Trollope emphasizes that Vector Sets provide “a more fundamental representation of vectors than in other vector databases.” He notes that Redis does not store the original vector because they “don’t believe the full vector is needed.” Instead, vectors are quanticized with methods like 1-byte quanticization or FP32 to binary conversion, achieving a 32-fold reduction in size. The specific quanticization approach varies with the use case.
Redis now boasts two complementary search capabilities:
According to Trollope, the LangCache “offers a hosted semantic cache via an API connection that enhances AI application speed and accuracy.” It functions as a REST API and incorporates advanced optimizations for reliable caching performance.
LangCache uses a custom Redis fine-tuned model and configurable search criteria, such as choice of search algorithm and threshold distance. Developers can generate embeddings through their chosen model provider, simplifying the management of models, API keys, and model-specific variables.
Moreover, LangCache manages responses so applications only return data approved for the current user, eliminating the need for additional security protocols.
Redis continues to expand its suite of AI developer tools and features, enhancing its AI offerings for developers.
Upcoming updates to Redis Cloud will provide facilities for building GenAI applications more effectively.
The Vector Set is set to be included in the Redis 8 Community Edition beta, launching May 1. RDI is currently in private preview, and interested parties can sign up for it. Redis Flex is available in public preview, enabling wider access.
For those keen on exploring more, Redis has released blog posts discussing LangCache, Vector Sets, and additional tools. Another post delves deeper into the intricacies of Vector Sets.
This new wave of innovations marks a pivotal step forward for Redis, as it continues to cater to the growing demands of GenAI model development, with a keen eye on improving speed and efficiency for developers across the globe.
