Qumulo Cranks Up AI-Powered NeuralCache
Qumulo has unveiled an innovative feature called NeuralCache to enhance the performance of its Cloud Data Fabric, pushing the boundaries of predictive caching. This new development is part of a continuous journey to improve data storage solutions for businesses navigating the complexities of both cloud and on-premises environments.
The Cloud Data Fabric (CDF), introduced in February, acts as a central repository for file and object data, featuring coherent caches at the edge. This system relies on a distributed file and object data storage cluster that integrates seamlessly with various platforms, whether vendors or public cloud infrastructures. The harmony between core and edge sites is maintained through sophisticated mechanisms like file system awareness, block-level replication, distributed locking, access control, and comprehensive logging.
NeuralCache embodies a breakthrough in data caching, utilizing a set of supervised AI and machine learning models to dynamically optimize read and write caching. According to Qumulo, it is designed to deliver extraordinary efficiency and scalability for cloud and on-premises environments alike.
CTO Kiran Bhageshpur explains: “The Qumulo NeuralCache redefines how organizations manage and access massive datasets, ranging from dozens of petabytes to exabyte scale. It adapts in real-time, considering multiple dynamic factors such as users, machines, applications, date/time, system state, network state, and cloud conditions.”
Qumulo highlights NeuralCache’s ability to “continuously tune itself based on real-time data patterns. Each cache hit or miss refines the model, boosting efficiency and performance as the system is engaged by users, machines, and AI agents.”
The technology “intelligently stacks and combines object writes, which minimizes API charges in public cloud settings while optimizing I/O read/write cycles for on-premises deployments. This results in significant cost savings without compromising data durability or affecting latency.”
The NeuralCache software also “automatically propagates modified data blocks in response to any writes across the Cloud Data Fabric,” ensuring that “users, machines, and AI agents always access the most current data.”
Bhageshpur adds that this innovation “enhances application performance and reduces latency while ensuring data consistency. It’s a game-changer for industries that depend heavily on data-intensive workflows, such as AI research, media production, healthcare, pharmaceutical discovery, exploratory geophysics, space and orbital telemetry, national intelligence, and financial services.”
Qumulo posits that NeuralCache excels at scales ranging from 25 PB to multiple exabytes, “learning and improving as data volume and workload complexity increase.”
This predictive caching software was originally part of the February CDF release, but as a Qumulo spokesperson shared, it wasn’t fully operational at that time and was referred to generically as ‘Predictive Caching.’ Since then, customer testing has shaped its evolution, culminating in its formal naming as NeuralCache.
In an interesting parallel, high-end storage array provider Infinidat also has a similarly named caching feature based on its array controller’s DRAM. Previously, it was described as a system where “data is prefetched into a memory cache using a Neural Cache engine with predictive algorithms.” This approach allows over 90 percent of data reads to be fulfilled from memory rather than slower storage drives.
Despite the similarity in naming, Qumulo’s NeuralCache is distinct and independent from Infinidat’s patented Neural Cache technology.
Qumulo’s NeuralCache is available immediately, integrated into the vendor’s latest software release. Existing customers can seamlessly upgrade without experiencing downtime. Discover more details here.
