This week’s Tools Spotlight interview is with Stephen B. Streater, founder and CEO of Forbidden Technologies. As pioneers in Web-based editing, their FORscene technology has become the world’s largest Cloud video post-production platform.
Out of all the companies with industry-standard video editing apps, only Avid has managed to release a Cloud-based version – and this only recently. How is it that Forbidden Technologies has been able to get so far ahead in this space?
Many companies have tried to use a readily available web video streaming technology, such as MPEG, to base their editing system on. The eight year lead Forbidden has in its frame accurate cloud post-production platform shows just what an obstacle it is to try to fit together pieces designed to solve the wrong problem.
When Stephen floated Forbidden Technologies in February 2000, we already had software which would easily transfer to the Web. Stephen Streater, CEO of Forbidden, was one of the pioneers of NLE, with a launch at IBC in 1990. Whereas competitors like Avid used hardware for their video compression, Eidos used dedicated software technology for its video and audio codecs.
Forbidden launched its “Playerless” video streaming in 2001 – years before the likes of Flash had video. Forbidden also launched streaming of live video (with a new Java player) in 2003.
FORscene, was launched in 2004 – a simple but frame accurate video editing system which ran entirely in the cloud – with a cross platform Java client which ran from a web page without any installation or configuration. The core to this system is the efficient technology which can handle real time transitions, effects, titles and multiple audio tracks in a Java applet.
Most editors would love to ditch the commute and work from home like web designers. But we have to work with huge files in networked environments. How does FORscene technology address these challenges?
The FORscene front-end works with a proxy video. The codec is designed specifically for editing, giving the illusion of instant access to any point in the video with minimal downloads. This is made possible by clever use of the video codec design. Those with a FORscene Server can access the video locally without waiting for the upload. The video is also accessible remotely: the Java front-end instructs a remote Server to prioritize those frames it is likely to need ahead of time, so they are already there when they are needed.
Many professionals in broadcast still finish on a high end system, such Avid or FCP 7. In typical broadcast workflow, FORscene is primarily used for review, logging and rough cut editing, ending with an AAF or XML export for finishing on these high end systems.
In the fast growing market for web video, editors finish entirely on FORscene. The publishing process uploads the full res sources – but only those frames which are actually used in the final video. This can be as little as 1% of the total source material. If a video is reversioned, only the additional source material is uploaded, making best use of available bandwidth.
The proxy quality is ample for most professional uses, with FORscene handling about 12,000 hours per week of professionally shot video, and over 2,000,000 hours of professionally shot video so far. This makes FORscene by far the world’s largest Cloud video post-production platform.
The creation of the lossless codec Osprey was a big step forward for FORscene. How did that breakthrough come about?
Although superficially straightforward, video editing is a large field. Forbidden has an extensive pool of technology available to address different markets. Professionals logging a 7 camera multicam have different requirements to a consumer shooting – and publishing – 1080p from a mobile phone.
Forbidden introduced lossless compression as a way to address the finishing market.
In practical trials, it turned out that a lot of the data rate was noise from the original MPEG compression. Adding a small element of “loss”, and using fewer corrections to Forbidden’s internal model of the video, cut a big percentage of the data rate and a big majority of people comparing the output thought the “lossy” version looked better. This all boils down to the video from the camera being already “lossy” in a typical modern workflow.
In practice, there is a much more popular workflow: the option introduced at about the same time of editing proxy quality and then publishing from original sources (which are, by definition, lossless) – though maybe not as technologically impressive.
How do the components Blackbird and Impala fit in?
Blackbird 5 is the current video codec. It has many features designed for internet editing, such as consistent quality, fast random access, graceful degradation on erratic internet links, low CPU requirements to allow up to 9 camera multicam in Java in a web browser (and mobile editing and review). On a more technical level it has 2×2 motion blocks with independent motion vectors, allowing moving video to be represented efficiently without making it all blurry.
Impala is the current audio codec. This is also efficient – allowing up to eight concurrent audio streams with audio levels and stereo positions to be decoded and mixed in real time – along with all the other tracks. The original audio compression has an additional loss free layer, giving it excellent datarate for the quality.
Many editors will be surprised to learn that FORscene supports multicam. Talk a little about that and how it has been used by content creators.
FORscene is used a lot for reality television, where there are usually multiple cameras. And as FORscene has always allowed editing and publishing while content is arriving, it is no surprise that one of FORscene’s big growth areas is sport, another area where multicam is common. Historically, the obstacles to multicam in the cloud have been two-fold: shortage of CPU and shortage of datarate.
When we were working on the Android app, we discovered that the Java-like language which Android runs was about 20 times slower than the Java on a laptop. We then reworked some of the technology to improve its efficiency even more. The direct result was a consumer version, Clesh, which could work on a 1GHz ARM chip found in mobile devices – a fraction of the computer power of a modern Intel CPU.
The flip side of this work was that, on a multicore Intel computer, such as my relatively ancient Apple Mac, you can run up to 9 cameras as a multicam. Here is an example of some of my 30fps 5D Mark II content (it’s worth expanding this to 720p if it starts small).
The other obstacle, internet speed, is also becoming less of an issue. All our biggest production companies have enough bandwidth to edit multicam directly from the cloud. But the FORscene Server sidesteps this problem completely. In a typical multicam workflow, the ingest Server is in the same building as most of the the loggers and editors, so they get their content over the fast local area network (LAN). As usual, everyone has different infrastructure and workflows, but FORscene’s flexibility allows it to fit in.
Network speed and reliability are also big challenges when working with proxy files on the Web. What is the FORscene approach to these issues?
FORscene was launched in 2004 – 8 years ago. The Internet is now 100 times faster. So network speed is not really an issue for us, even when companies have dozens of people using FORscene at the same time.
The network does seem to be pretty reliable, as a result of its famed nuclear-proof ability to re-route packets through working switches if there are any failures. The weakest point is probably at the user end, but in an emergency, you can up sticks and move to your local cybercafé to carry on while they fix it. You can even set up your mobile phone as a WiFi hotspot and use that.
The mains electricity supply is probably a bigger risk these days, and no one is suggesting going back to film to avoid power cuts. So if you are paranoid, use a laptop with a mobile backup. It’s fair to say that this is not an issue in practice.
It seems like every week we hear about a major online data breach. How does FT protect uploaded content from unauthorized access and also insure against data loss.
Yes – it’s a good headline. With a billion people using cloud services every day, we only hear about one online breach a week. How many of these are not even related to cloud services, like some random virus or worm infecting internet connected PCs? And how many physical security breaches (burglaries) are there a week? The reality is that, just as with physical security, it is impossible to absolutely guarantee protection against data loss on the internet. This applies to every PC, Mac, smartphone and tablet. But we can all take sensible precautions.
FORscene has a Java front end, and Java is relatively secure with Oracle patching bugs quickly. Even Windows has timely security patches these days, though users should make sure their Windows / OS X / Linux version is supported and keep up with the latest patches.
Access to the system is password protected, with https used to secure log in. If https is cracked, there will have much bigger problems, as this is the banks use for their online security. These days users are much more aware of the need for secure passwords.
Most users see only proxy quality, with HD essence only accessible to users who need it. To an extent, you have to trust your editors – they could still film the video with a cameraphone – but at least it will only be proxy quality.
Our standard procedure is to make copies of the data at three different physical sites, two on the Internet Backbone for rapid access. If there is a fire at one site, they data is still available elsewhere. Forbidden does tell users to keep copies of any material they need and not to rely on the FORscene system for archive. The typical workflow involves ingesting material from another system for editing, so users just keep the originals. In practice, the use of RAID storage with redundant disks and multiple sites means that a data loss through disk failure is unlikely.
FORscene supports EDL and XML export into industry-leading editing systems like Avid and Final Cut Pro. How well does the metadata from FORscene projects transfer over to these NLEs?
These days the logging data does transfer across. This includes frame accurate start and end points. In Avid, essence markers are now created automatically. In practice, Directors often searches through metadata in FORscene, with its fast incremental search, and just tells the Avid editor where to find the material they are looking for.
Is there a way to integrate the FORscene project into an off-site archive system like CatDV or StorageDNA?
Yes there is – provided you are Forbidden! Discussions with archive providers are underway. Many customers have CMSs which FORscene integrates with. For some reason, this came up first – perhaps because the biggest need was for a cloud finishing and publishing system.
What is Clesh and how is it different from the main FORscene product?
Clesh is the consumer brand. The biggest difference is that FORscene allows more professional features, such as EDL output, multicam and review on iPad. Clesh is aimed at prosumers and consumers and starts in Storyboard view with 43 fewer buttons visible than the standard Timeline view in FORscene.
Clesh on Android doesn’t have a logging view at all – not may people will be logging on a phone without a keyboard!
Perhaps the biggest difference is that FORscene users are pros who have a business – and a business model. Many of our improvements to FORscene are inspired – and paid for – by FORscene professional users. Clesh users benefit from these when they are released.
Sales of Clesh for Android reached a new record last month. Why no iOS version?
Android is a lot easier to develop for, as you can do whatever you like. For example, the Java-like syntax used by Android apps allowed vast swathes of the Java web applet to be recompiled without modification. For a long time Apple didn’t allow you to use Java as your programming language on their platform.
On Android, you can also publish and distribute independently of Google if you want to. Even Android itself is open source. In contrast Apple appears to see their app writers as an extension of Apple, furthering Apple’s agenda. Apple decide what apps you can and cannot publish, for example.
Google wants as many good companies and apps as possible on their platform – and even want people to try and produce better software than them.
Apple doesn’t like competitors, particularly in key strategic areas. Internet video, video editing, the cloud, codec technology, mobile software – these are all key strategic areas. Apple sees app customers as their customers, and restricts knowledge about the end users available to developers. This doesn’t work so well in our cloud service. And of course, Clesh runs on PCs, Macs, Linux – and Android. The cloud is a real threat to Apple’s hardware business model. Why buy an iPad and a Mac when you can run the same software on a $100 Android device and a $300 Linux box through a browser? There are of course reasons to do so, but Apple users can be surprisingly unaware that there are often better alternatives out there. Perhaps Apple likes it this way.
So Forbidden would prefer to know from Apple that they would not try to ban the Clesh or FORscene service from their platform before investing heavily in a superior cross platform solution for it. Or maybe Apple could invest in and license a superior version for their platform. In the mean time, we will continue to improve our service on the many open platforms available.
What kind of feedback and feature requests have you gotten from FORscene and Clesh editors lately?
Some of the important ones are:
- Different fonts for titles (recently in use)
- FORscene Server (download / upload proxy and local compression to accelerate uploads) for PC (recently in use)
- FORscene Server for Android – prototype available: allows editing before the video is uploaded into the Cloud
- Rotate – consumers particularly want this, shooting video as they sometimes do in portrait - and professionals who use consumer shot content also would like it
- Integration with work order management systems for big scale use – work ongoing on this
What are the biggest challenges moving forward for remote, collaborative content
Users getting to grips with its potential. Cloud infrastructure is a lot more flexible than desktop-based workflows, and people are still working out how they can make use of them. The technology always moves a lot faster than the installed user base.
Is there anything that I didn’t ask you that you want to mention?
It’s good to have licensed FORscene to YouTube at the end of last year. Working with Google is fun.
Also, we are a public company in the UK, so publish as lot of information. This is accessible on the Forbidden website