Budgeting restoration job

Fateh Shams Sep 21, 2016

  1. Hi,

    I need some help trying to calculate digitisation and restoration job.

    We are taking care physical video archive for a company (cant mention the name here) and planning to digitise their video archive.

    We will start with the recent years video material and go back year by year. Source material is from HDCAM SR, HDCAM, Digibeta, DV/DVCAM/DvcPro, BetaSP, U-Matic, 1"C tapes. Also 16/35mm film but that is another part.

    All the tape decks are available in-house. We are planning to invest in Phoenix system. Digitise the whole tape to BMD UltraStudio as 10-bit Uncompressed SD or HD file. The newly shot material only needs de-interlace (maybe some de-noise, crop the black edges and keep it 720x576 for SD and 1920x1080 as HD. Archive it as UnCompressed to LTFS LTO-7 and keep the ProRes 422 HQ on Online storage. (We already have workflow for logging and archiving, there we are getting recent material as digital files from Production houses, mostly has ProRes 4444 or 422 HQ, I know this is not future proof but that is what we are getting from the production houses)

    Lot of films on Digibeta tapes are once transferred from 16/35mm film to 1"C or other analogue formats until those gets on DigiBeta. Some films are still on 16/35mm rolls left in the archive and for those we will look in future the scanning & restoration possibilities.

    Digibeta tapes with sources coming from 16/35mm films needs more restoration work. Everything from Dust/Scratch/Flicker removal, de-grain, stabilisation, fix the joints and also digital errors, like drop frames, line fixes and color fix.

    My Question is:
    Can someone, already working with Phoenix system, can give me estimated time for this kind of work. Like how much 1 min SD material can take to do the treatments like that and make it ready for re-mastering. Is 1 min film is equal to 20-30 min restoration work 50-60 mins or more? Also what is the estimated charges for this kind of work? These are the industrial films and some TVC back from the days. TVC might take more time in restoration as those are more important to the customer.

    The investment will be around 70K EUR for the setup and this project will continue 3-5 years.

    Any other suggestion will also be appreciated and feel free to ask question. English is not my native language but hopefully I have been able to explain the workflow.
     
  2. We don't use Phoenix (yet), but have extensive experience with MTI, Diamant and PFClean. I can tell you that for an HD feature film (figure 90 minutes), one person working 8 hour days will take about a month to do frame by frame cleanup. This assumes manual dustbusting and some scratch removal, and a reasonably dirty film. That breaks down to about 4.5 minutes per day, assuming you're working with 24fps material. If you're dealing with NTSD SD, it's more like 3.75 minutes per day, because you have that many more frames. Not really a factor for PAL SD. These numbers vary, of course, depending on the extent of work to be done. We don't give estimates to clients without seeing the film first, because every project is different. It's virtually impossible to do an estimate in a generic way, but our experience has been that we can do a feature in about 4 weeks with one operator working only on that.

    That being said, I would strongly urge against doing restoration work on old SD transfers of film. You're wasting time and money if you do. For one thing - it's SD and who wants that these days? You end up doing almost as much restoration work, which will likely need to be redone later if the film is re-scanned. I would instead budget on getting better scans at higher resolution of the film material (2k at minimum), at its native aspect ratio - not HD.

    In terms of digitizing old tapes, if your goal is to create digital preservation copies, I wouldn't deinterlace or process the video on ingest. The generally accepted best practice among archivists is to try to capture the tape as close to the native format as possible. So if it's interlaced tape, capture it interlaced and store that. Make deinterlaced or cropped versions from those for access. Uncompressed files are going to add up in size, especially if it's a sizable archive. It's a good format to use, but you might also consider ProRes 422 HQ, since the files are smaller, the compression is very light, and it's 10 bit.
     
    Jason Myres and Marc Wielage like this.

  3. True, those films which we know are still available on 16/35mm film we will not restore at the moment. But there are dozens of films which are unique and we might not able to trace back the master film prints. Those will be restored from the tapes.

    My initial plan was to store processed Uncompressed + ProRes version on LTO tape and only keep a copy of ProRes version on our online disk storage for easy access. I will reconsider about storing the unprocessed uncompressed version also on LTO.

    What is good choice of file format? 16-bit TIFF sequence (without LZW) and audio files as WAV?
     
  4. I usually steer people away from 16bit TIFFs. they're a nightmare to work with. Once you start getting into HD or bigger, the file sets are just tremendous and very clunky. Also, fewer applications support TIFF sequences than DPX, in my experience. You're probably fine with 10bit DPX. Considering the source is video, you're not gaining anything by moving to a 16bit file, but you're definitely making life harder for yourself!

    For audio, we usually do 96/48 PCM - WAV or AIFF, they're effectively the same thing. For tape formats that have digital audio, we just use whatever the tape format used, as far as bit depth and sample rate.

    -perry
     

  5. So that 4 weeks / 8hr daily restoration, does it also include the final color correction?
     
  6. No. Just restoration work. Color correction is a whole 'nother can of worms!
     
    Marc Wielage likes this.
  7. What they used to say about dirt removal at Lowry and at Cinesite was: one you remove all the fairly large, visible 3-pixel and 2-pixel dirt, suddenly the client starts noticing everything that remains, no matter how small. So the company had to invoke size limits on what was worth fixing and what was not. There were some clients for whom cost was no object (like Disney), and six months for dirt removal was not unheard of on some film restoration projects. Those were termed "pristine," meaning no dirt allowed at all.

    My personal rule was absolutely no dirt allowed on the actors' faces, ever. That's a big no-no. I get that for a modest restoration job, it's never going to be perfect. Chances are, even with a 2-week or 3-week cleanup, it's going to have far less dirt in it than any projected print ever made in history.
     
  8. I told a client I'd clean just the worst bits of dirt off their film (16mm feature which had got messed up in the neg cut, been rescanned with some automatic dirt removal applied and still looked bad). Once I'd painted the worst bit of dirt off every shot, I reviewed it and simply saw the next worst bit of dirt - I realised I could have gone on for ever! Two or three passes and we left it which in the end was a very good comprise. Luckily I was able to work on it between other jobs so it wasn't solid weeks of paint
     
    Marc Wielage likes this.
  9. One approach a couple of our clients take is to set a flat budget against our hourly rate for restoration. They have us do the first two and last two minutes of each reel of the film, since those are typically the dirtiest. Whatever is left over in the budget then goes towards really big stuff in the middle of the reels. If there's any left, general dustbusting until the budget is exhausted.

    One of the really hard things about restoration, if you're a little OCD about dirt on film like we are, is stopping yourself from cleaning everything while you're in there. It seems like it's only going to take a second, but if you're talking about 120,000 frames, one extra second per frame works out to 33 extra hours on the job...

    There is a bright spot though - unlike 10 years ago when automated systems were more trouble then they were worth, auto dirt fixes on modern restoration software is much better at avoiding false positives. You have to go into an automatic pass with the understanding that it's not going to catch everything, nor should you try to do so. Set it up to catch *some* of the dirt and then do the rest manually. Some clients don't want any automatic cleanup at all, which is fine. But you have to pay for all that manual work. We're finally at a point where a reasonable balance has been struck, though it's still a ton of manual work, even if you do use automatic passes.
     
    Marc Wielage and jamie dickinson like this.
  10. I have not been at IBC but heard that Phoenix launched new dust and dirt took which should be far more advance than anything in the market. They said that users of other restoration software claim that they can achieve the desired result with new algorithms in minutes vs 4 hours with their existing tools.

    Anyone else knows more about the new dust & dirt tool from Phoenix?
     
  11. DigitalVision showed this at NAB as well, and it was very good. But the problem with automatic cleanup tools is that there is simply no way for a computer know whether certain things are actually part of the image or not. For example, there might be a shiny object that gets a glint of sunlight for one frame. Most automatic algorithms will treat that as a transient artifact and remove it. How is the computer supposed to know that that thing that lasts for just a frame is actually supposed to be there? It can't. That requires a level of intelligence and an ability to understand visual context that software can't handle at this stage.

    What DigitalVision showed was *really* impressive, but it will still require extensive QC to make sure there weren't false positives. This is one of the big problems with automatic cleanup - you're either spending a ton of time doing manual work or you're spending a ton of time QCing an algorithm's work. I'd personally rather do it manually from the get-go, because that way you know exactly what you cleaned, and you're QCing the work as you do it. Then you just need to watch it through looking for potential artifacts, without having to be concerned about having removed things that shouldn't have been removed.

    -perry
     
    Marc Wielage likes this.

Share This Page