With rolling blackouts predicted just as the post sector hits its busiest period, it’s time to review your disaster recovery plans
With the post-COVID backlog barely cleared, a winter of rolling blackouts looming and a newly distributed workforce, this year looks set to be a challenging one for UK post houses. One of the most important steps you can take to protect your business and reassure your clients is reviewing your disaster recovery (DR) and business continuity (BC) plans. Pre-sales consultant Phillip Boettcher is here to help…
First of all, what’s the difference between disaster recovery and business continuity planning?
Phill: Disaster recovery is the process of regaining lost data, media and work after an outage or system failure. Your disaster recovery plan should explain the steps you take to protect your assets, how quickly you could recover them if something went wrong (known as your ‘recovery time objective’ or RTO) and how much data you can afford to lose (known as your ‘recovery point objective’ or RPO). If you can afford to lose one day’s work and it’ll take you 12 hours to restore from a backup, your RPO is 24 hours and your RTO is 12 hours.
A business continuity plan details all the steps you take to help people keep working if your facility is compromised. For example, do your editors all have thin clients at home so that they can work remotely if your office catches fire? Do your admin team use cloud-first software so they can log in from anywhere and continue managing clients if your own network goes down? Questions like these should be answered by your BC planning.
How often should businesses review these plans?
In an ideal world, you should be reviewing disaster recovery and business continuity plans every six to twelve months. However, many companies we work with will make an ad hoc plan when their facility opens and fail to keep it fit for purpose as the company grows. In fact, many established companies only dusted off their plans once the 2020 lockdown forced their hand.
Why is it so important to address these issues now?
1. It’s front of mind for your clients – and Ofcom.
After a series of high profile disasters, Ofcom launched an investigation into – and recently issued a report on – the lack of resilience in broadcasting services. It specifically mandated that all DR systems should support video, audio and accessibility content such as subtitles, be regularly tested and be subjected to simulated emergencies to ensure their effectiveness.
2. We’re heading for an energy crisis.
The UK government are refusing to rule out rolling blackouts this winter, which means your system could be subject to a sudden or planned complete loss of power. Many facilities we speak to have a layer of legacy hardware somewhere in their technology stack that has worked for so long no-one bothers to test it anymore – which means no-one knows if it will turn back on following a power cut. Now is the time to test these systems, update elements that need to be updated and ensure the content that depends on them is securely backed up.
Relatedly, if you’ve never performed a controlled shutdown of your systems, now’s a good time to get that strategy in place, understand how long it will take and identify any weak links in your infrastructure that could cause problems in the event of a forced shutdown.
3. Your RTO and RPO are probably in flux right now.
We’re heading into the busiest time of year for post facilities. That means you’re handling more data than usual, are dealing with more assets at once, and are probably gearing up to add freelancers to your team. All of which increases the strain on your back up and recovery systems.
Let’s say a post house has decided that their RPO – the point up to which they can afford to lose their data, is one day. At the end of each day, the day’s work is automatically backed up to an offsite location overnight. In the event of a catastrophe, they can simply restore the last backup and voila, they’re now only a day behind.
As long as the backup solution works, that is.
During peak times, their network may not be able to transfer all the necessary data during the overnight backup window, leaving holes in their record and creating queues that prevent their most recent changes from being safely saved. The time to discover this has happened to you is not after a blackout; it’s now, while you have time to improve your throughput before those freelancers arrive.
4. Remote working complicates things.
Are your staff taking media home with them when they work remotely? Are they making changes to media locally before uploading it to your shared storage system? All of that needs to be accounted for in your disaster recovery plan – no matter how many locations you may now have data in.
5. Your post-pandemic workflow can be shored up.
We’re on the other side of COVID now, and a lot of the remote systems that seemed adequate when you just needed to find some way for people to work will no longer seem sufficiently robust.
This is an ideal time to revisit those systems and shore them up with more long-term remote and hybrid working solutions, which will improve the resilience of your facility and make it easier to ensure business continuity during the most high-pressure time of year. While remote working does complicate things from the point of view of backup, the slew of remote and virtual tools available now is a real blessing when it comes to business continuity, and it’s well worth exploring your options.
What are the consequences if companies don’t get their plans up to date?
If being publicly shamed by Ofcom isn’t motivation enough for you, try the oft-cited University of Texas study that found 94% of businesses that suffer catastrophic failure don’t survive. Or the Gartner research that found the average cost of network downtime was over $5600 per minute. Or the IBM report that uncovered 60% of data breaches impacted the cost of services to end clients – not something you can afford if you want to remain competitive.
Since the UK media sector was blindsided by the COVID pandemic, we’ve seen a massive increase in the number of clients who want to see detailed disaster recovery and business continuity plans before they’ll commit to handing over their data, and not having an up to date process in place can cost you work.
Even if you do manage secure jobs, it can’t be stressed enough that these assets, this data – that’s where your money is. It doesn’t matter how nice your premises are or how talented your team is if you don’t have these assets and can’t recover them.
So, how can disaster recovery be improved?
The first and most important step is to make it a frequent consideration and ensure you have time and budget set aside for regular testing and improvements. The second is to identify older hardware and weaker networks that could potentially cause trouble. You also want to think about exactly how redundant you need to be – can all your company data be backed up to one offsite location, or do you want multiple offsite copies? Should the systems at each of your locations replicate each other’s data? And, crucially, how will you transfer and restore this data?
A strong candidate for anyone looking to drastically improve their disaster recovery times in short order is Object Matrix’s MatrixStore and MatrixStore Cloud, which allow you to automate on-premise, off-premise and cloud backup so that you can guarantee the safety and accessibility of your assets. This is particularly useful for tier two storage, as it provides you with a simple, secure way to move projects which are completed but under review off your production storage without compromising the speed with which they can be recalled.
Our MatrixStore system is scalable, so you can flex up and down as your workload requires, and there is a great deal of redundancy built into each node of the storage architecture, so you’re protected not just against general failures but against failures within your storage system itself. But most impressive are MatrixStore’s multi-site tools, which allow you to synchronise two different Object Matrix deployments so that each system provides redundancy for the other. Any multi-site company running this system will have complete redundancy for any given site if they lose access to one location, and we can help you develop a public cloud or data centre-driven solution to provide redundancy in the event that you lose access to all your sites.
About Jigsaw24 Media
Jigsaw24 Media is a specialist division of Jigsaw24 and provides services and technology solutions to the media and entertainment, education and corporate sectors. It’s the only UK-based company of its kind that has in-house system integration capabilities, one of only two Avid Elite partners in both video and audio in the UK and Avid’s global partner of the year. Jigsaw24 Media’s team of industry-recognised experts design, deliver, integrate and support end-to-end solutions for some of the nations’ biggest broadcasters and facilities, underpinned by partnerships with over 30 leading technology vendors including Avid, Adobe, AWS, Nutanix and EditShare. With headquarters in Nottingham, an office and demo space at the heart of London’s post-production community in Soho, and a nationwide support team, Jigsaw24 Media provides local services on a national scale. For more information visit media.jigsaw24.com.
About Object Matrix
Object Matrix is the award-winning software company that pioneered object storage and the modernisation of media archives. It exists to enable global collaboration, increase operational efficiencies and empower creativity through deployment of MatrixStore, the on-prem, hybrid and cloud storage platform. Their unified deployment approach ensures content spans on-prem and cloud storage whilst their focus on the media industry gives them a deep understanding of the challenges organisations face when protecting, processing and sharing video content. Customers include: NBC Universal, TV Globo, Warner Bros. Discovery, MSG, ATP and the BBC, to name but a few!