Object Storage – Benefits in Media Workflows
As a concept, object storage is not new – as a company, Object Matrix has been developing it for over 11 years. But it is recently starting to get traction in the media and broadcast industry being slated as the next big thing.
So what is Object Storage, and why is it good for media workflows?
Traditional storage relies on a file system interface – that is you present your network attached storage (NAS) as a file system interface such as a drive letter on Windows or as a volume on Mac or Linux. But traditional file systems have scalability issues – an upper limit on file numbers (typically millions) and as you approach this limit then performance becomes an issue. Object Storage does not rely on a file system to manage the content under its control and so, in theory, there is no upper limit on file numbers. This in turns allows you store petabytes and beyond of storage with no loss in performance. An object is in fact just a digital asset such as a media file.
And there are additional benefits. A file system does not allow you to store metadata with content – it’s a limitation. However, object storage does allow this making it searchable and intelligent about its own content without the need for a separate media asset management system. Or when using MAM’s they can be configured to archive and protect metadata as well as the media files themselves. It is also possible to manage different objects with different data management rules such as a number of copies and replication rules.
Facebook and Amazon S3 are built on Object Storage principles – Amazon S3 has trillions of objects under its management which it provides in an online cloud type environment. Object Storage must allow for loosely coupled inter-dependencies – in other words having different users and applications being able to access the content without impacting on each other. The MatrixStore architecture virtualises multiple nodes of physical hardware to enable it to be used securely by multiple users, and with minimal impact upon each other. Also, the flexibility of an object storage architecture means that multiple storage policies can be enforced, e.g., for how long objects are stored, how many instances of data are kept and the location of those objects.
So why object storage for media and broadcasting workflows?
Simply put, the media world has large storage expectations. Media files, rushes, sequences, finished projects and ongoing transcoding and repurposing requirements not only dictates a platform that can manage big volumes of data, but also one that can be performant enough to serve up content in real time. LTO tape is a traditional archive medium of choice, but with its limitations, media organisations such as broadcasters, VoD companies, advertising agencies and post-production companies are acquiring additional disk based “nearline” storage solutions for storing media and other digital files as their main media repository. Disk allows random access, is quick and performance is not limited by the number of tape drives. Disk-based Object Storage takes that further by allowing a very expandable and scalable solution that can be searched, is highly resilient, easy to use and can be delivered with commodity hardware that is easy to support. And all at a price point that is compelling.
Additional storage benefits of a disk based system such as the MatrixStore from Object Matrix:
Self-Healing Functionality in the event of any hardware failure providing 99.999% uptime.
No “Lock in” to specific hardware – as you scale (as that is the point!) over time, grow your storage with the latest and greatest storage technology to expand your platform.
Authenticity check with “checksum”, such as MD5 or Adler32, to ensure a bit-for bit copy when ingesting content from another platform
Workflow Integration – proven integration with various technology platforms such as Avid and their Interplay MAM/DAM platform, Adobe, Apple, Cantemo, CatDV, Harmonic, Glookast, MOG, Cambridge Imaging Systems, Aspera, Signiant, Masstech as well as an array of other transcoding, MAM and play out partners. Inter-dependency as described above allows sharing of applications on the same physical hardware without impacting on the security and data management needs of the other workflows.
Various client tools including a ‘virtual’ file system for Windows, Mac or Linux (MXFS), an SMB interface, or DropSpot, our ingest and search tool. A file system can be assigned to each workflow and mounted as a separate ‘logical’ file system from the same physical hardware.
You can have a very scalable single file-system to petabytes and beyond if you desired.
Easy to use and administer.
Easy to add storage and grow (In less than 1 minute) with re-balancing of content for improved performance.