Recent advances in machine and deep learning are moving GPU driven artificial intelligence applications from R&D labs and Silicon Valley startups into the computing mainstream. AI is presenting a new set of storage challenges that IT organizations have never had to deal with.
Customers need to keep GPU servers fed with more bandwidth than what a NAS can deliver, causing them to think that complex parallel file systems are the only answer. On the other hand, file accesses are small and random in nature, making AI the killer use case for flash storage – but parallel file systems and burst buffers were never designed for providing random read access to petabytes of flash. As AI becomes democratized, niche HPC storage products are difficult for enterprise IT organizations to adopt and operationalize on a broad basis.
VAST combines the single-host performance and scalability of a parallel file system with the simplicity of an all Flash NAS appliance. By marrying the performance of Flash with economics that are otherwise only found in HDD-based archive storage, VAST enables AI applications to consolidate infrastructure and accelerate the training and inference time.
VAST’S KEY BENEFITS
PARALLEL FILE SYSTEM SPEED AND SCALE
Scale to TB/s, millions of IOPS; 100GB/s+ per AI client over Ethernet or InfiniBand
Turnkey Universal Storage appliance - with no client SW dependency or complexity
Radical flash storage economics to make flash affordable for all AI datasets
Breaking Decades-old Tradeoffs
THE PYRAMID IS DEAD.
For decades, storage practitioners have been trained to tier their storage infrastructure in order to save money and to keep their largest datasets on slow, archival storage. Today, the rise in adoption of new machine and deep learning techniques require training on vast amounts of data, where data needs to be fed to farms of GPUs with maximum throughput. Training algorithms only get more effective as they are exposed to more and more data, thereby rendering the classic storage tiering model obsolete in the AI era.
A New Type of Storage Architecture
VAST Data Universal Storage.
By applying new thinking to decades old storage problems, VAST has broken the long-standing tradeoff between storage performance & the cost of capacity to make it possible to simplify and accelerate ML/DL pipelines.
Exabyte-Scale All-Flash Namespace
Millions of IOPS from cost-effective QLC flash, enabled by Storage Class Memory technology.
RDMA Client Access
NFS over RDMA provides over 10GB/s of mount performance over Ethernet or IB; scale multiple mounts in a single client to drive 100GB/s+ of single client bandwidth.
VAST has pioneered many innovations to democratize flash: QLC flash translation, low-overhead erasure codes and groundbreaking global data reduction.
Multi-Tenant Training and Inference Infrastructure
VAST server pooling capability provides dedicated QoS for competing applications.
One, simple-to-manage scale-out file system appliance, no need for complex client-side file system client software.
With no east-west cluster traffic, VAST’s DASE architecture sets a new standard in scaling to the needs of massive ML/DL server farms.