Cloud Storage Innovation of the Year
What exactly is novel about your product?
LucidLink is transforming the cloud file services paradigm. LucidLink Filespaces™ has been designed to address the limitations of NFS and SMB protocols. It’s a modern file system that works equally well over LAN and WAN, has a high-performance concurrent streaming architecture, reduced network chatter and a robust security model, and is compatible with all major operating systems (Windows, Linus, macOS).
LucidLink enables object storage to be deployed for file-based, primary storage applications. Moving beyond backup, archive, and re-factored native S3 applications, the combined solution extends object storage to general-use storage for servers, application data, and even user directories. Filespaces provides a uniform shared, global file system namespace across any device connected to the Internet. It is a SaaS solution that delivers the intelligence to make any simple object storage act as a high-performance file system volume shared and accessible from anywhere. Developed to offer a near-local user experience, even when accessing data over distance. Filespaces functions like network attached storage (NAS) or file server, but both the clients and data repository can be located anywhere. All authorized client devices can see and access the same shared data set, irrespective of their operating systems or geographic location, as long as they have network access to the S3 object store.
Filespaces uses any S3 compliant object storage as a back-end repository and works with any cloud (public, private or hybrid), any file type, and all major operating systems to make the cloud perform like a local hard drive extension. With LucidLink, companies with distributed workloads can access terabytes+ of data instantly from any location and any machine – no matter how large the file or how remote the access point. Filespaces is an innovative alternative to today’s dominant cloud storage solutions because it’s cost-effective, delivers virtually infinite capacity and best-in-class security, with zero operational overhead. You might even say it’s the next-generation of cloud data center infrastructure.
How does the product break the conventional ideas or processes in the field?
In the age of digital transformation, data grows exponentially, as does the need to access it instantly across locations. It’s a dilemma that the cloud was supposed to solve, but even in the cloud, distributed remote data access can be time-consuming, cumbersome, and duplicative. That’s because up until now, cloud storage and file services solutions have re-packaged technologies initially designed for on-premises storage and run them in the cloud. LucidLink breaks with convention by targeting the source of the problem – distance and latency – thereby allowing customers to consolidate their data in a secure, cloud-based SaaS platform that provides high-performance, streaming data access with no need to download and synchronize files when accessing from remote or globally distributed locations. LucidLink moves beyond backup, archive, and re-factored native S3 applications by transforming object storage into general-use storage, using it as if it were a local, shared volume.
Three unique innovations underpin our high-performance offering and distinguish it from other solutions in the cloud storage ecosystem. First, we created a log-structured, cloud-native, distributed file system that extends the value of object storage. By using the object store in a unique data layout – as a block-level, scale-out storage platform – LucidLink can deliver higher performance and infinite capacity. Next, we developed a specific protocol – using parallel TCP streams; local write-back caching; in-line compression; and a proprietary, adaptive algorithm for advanced data prefetching – that solves the problem of distance and latency in cloud environments by removing up to 80 percent of the traffic between applications and files. This protocol separates metadata from file data, synchronizes the metadata locally, and enables high-quality, on-demand data streaming from any location as users or applications request access. Finally, remote distributed workloads don’t exist without concern for data security, so LucidLink took a novel approach to data confidentiality in the cloud – one that minimizes the amount of trust required by all entities involved in storing, managing and transferring data. LucidLink’s “zero-knowledge” encryption model uses robust end-to-end, full-system encryption where all data is encrypted on the customer device and remains encrypted both in transit and at rest with only the customer in possession of the encryption keys – neither LucidLink nor the storage provider (e.g., Amazon S3) can “see” the data. These factors and more put LucidLink at the forefront of cloud storage innovation and enable us to provide seamless, secure, on-demand data access, no matter how distributed or remote the workforce, or how large and dense the data file.
How does it go beyond marginal improvements on something that already exists?
Over ten years ago, Dropbox and Box took a big hammer approach to the problem of remote data access for distributed workforces. The idea was simple, put a copy of the file wherever you need to access it and keep changes synchronized across all devices. This worked for users with small-sized files and data sets with casual sharing requirements, and both companies achieved success. However, as data sets and file sizes grow, this approach doesn’t scale. Users still need to download and sync files, consuming precious time and resources. Cloud vendors attempted to fix the problem with native solutions, but these don’t work for hybrid, multi-cloud, or mixed OS environments. Storage gateway vendors also offered options, but these are expensive, complex, and break down over multiple locations.
By design, LucidLink goes beyond merely making marginal improvements to existing technologies. Specifically, Filespaces addresses the limitations of both the NFS and SMB protocols. It is a modern cloud file system that works equally well over LAN and WAN, has a high-performance concurrent streaming architecture, reduced network chatter, and robust end-to-end encryption (AES-256 in GCM mode) that protects data at rest and in flight. LucidLink does away with synchronization to stream data on-demand directly from the cloud – instantly, regardless of the operating system or device. Filespaces maintains the shared data store in the cloud, with frequently accessed data cached locally to deliver the best of both worlds in terms of performance and flexibility when accessing TB to PB-range volumes. Filespaces untethers devices from their location and allows the same uniform experience, whether in or outside the office, on-premises or in the cloud. LucidLink’s software-only, cloud-first approach significantly expands the use cases for cloud file systems, provides much better integration with the host operating system, and eliminates capital expenditures. The result is the next generation of cloud data center infrastructure that delivers an all-new level of speed, functionality, and cost savings.
How do customers benefit from the product or innovation?
As organizations move toward digital transformation, they seek to adopt file storage and service technologies that are agile, secure, easy-to-manage, scalable, and cost-efficient. LucidLink Filespaces™ delivers this and more. Filespaces is a no-compromise solution for leveraging file services for private or public cloud storage. It provides unlimited capacity that requires no infrastructure and works with any S3 compliant storage provider, any file type, any device, and all major operating systems. With innovative functionalities such as direct read/write access from the cloud, snapshot support for backup and security, persistent caching and prefetching, user access control, and global file locking, customers benefit from streamlined collaboration and workloads, time and cost savings, reduced to no data duplication, and tighter control over data storage, security, and device and sovereignty mandates.
LucidLink has a number of use cases in M&E and AEC industries and the US Government as well as partnerships MSPs and distributors.
John McNeil Studios (JMS), a California-based creative studio that works with media-rich data files, is already benefiting from the Filespaces platform. JMS cites benefits such as improved remote access to large video files, costs savings via better remote access for freelance workers, and further cost savings because Filespaces works on their existing storage area network (SAN).
A US government agency uses LucidLink for a common data repository for distributed workflows. According to one senior analyst at the agency: “Retrieving file type data from S3 is a nightmare. LucidLink offers an elegant way to get around all that. With LucidLink, I can share things over much further distances which is impossible to do any other way. This is a complete game-changer for the way we handle video. We can never go back.”
LucidLink’s partners – managed service providers and cloud service providers – are also benefiting from the ability to offer a responsive, available, durable cloud-based platform that’s low-cost, high-performing, secure, and flexible enough to support multiple operating systems within the same installation.