That Boring File Management Stuff….


So the PA project(s) are coming along, but I think I’ve decided to wait to talk about the project(s) because they’re happening so fast. Electricians were in last week getting the power infrastructure tweaks started. This week and next we’ll be pulling cable in anticipation of Phase 1 the week after Easter when the new loudspeakers fly. But I digress…

As some of you might know, we recently moved the North Point Online webcast from Buckhead Church to my campus. The move brought with it some minor changes in how we are handling multi-track recordings on Sundays. Previously, we were using our studio primarily for capturing the audio side of the service with a mix done secondarily in the studio to feed our tapes, resources, the podcasts, etc. NP Online, however, has transformed our studio environment into a priority mix room for the NP Online mix.

We’ve been working in the studio mixing exclusively in Pro Tools for a few years, and we’ve opted to maintain that setup for now. However, since some of the content for NP Online is targeted specifically towards those joining us online, multi-track recording in the studio doesn’t necessarily fit into the workflow the same way. Plus most of the stability issues we’ve encountered mixing in Pro Tools seem to happen while we’re recording and mixing simultaneously. Consequently, the decision for now has been to no longer record during the webcast. We might track during rehearsals so we can go back and tweak during downtime leading up to the webcast, but for the last few weeks there has been zero recording happening in our studio during the webcast. No recording in s a studio, go figure.

One of the consequences of terminating recording in the studio has put a greater recording burden on FOH. Our FOH engineers are no longer just mix engineers, they’re now recording engineers as well. Of course, this isn’t something entirely new as we’ve been recording everything at FOH since the Venues were implemented. However, these FOH recordings have served largely only as safety copies in the past, but now they are the master recordings.

This has created some challenges since there is a need every week to get those master recordings back into our studio for any post and sweetening needs for podcasts, and master CD’s. Plus there might be songs that need mixing for DVD release or a remix of a message for North Point TV or something else we haven’t even conceptualized at this point. The hard part has been that a typical Sunday morning session runs 80-100 gigs per auditorium–remember, we’ve got two. In the grand scheme of things, the size isn’t really a problem with the cost of disk space at an all-time low, however transporting it across our existing FOH network infrastructure can be a time consuming process. Add to that the fact that the post engineer probably won’t need the entire session on Monday morning; he might only need 6-10 tracks from one service. But, I think I’ve found a solution that’s going to work nicely for us.

This was largely inspired by the archive setup that the production guys at our Browns Bridge campus devised. A few weeks ago they turned me on to some hard drive docks that you can plug a standard 5.25″ SATA drive in. There’s no need for a drive enclosure or caddy’s or anything else. You can literally just stick a drive into a slot on the top of this thing. Doing a little more research, I found a newer model by NewerTech over at Other World Computing. I started looking around because the dock that BBCC is using wouldn’t be compatible with our existing FOH setup, but also because it didn’t have the Oxford chipset that Pro Tools likes if you’re going to record on the drive. I’ve used OWC for probably ten years to purchase Pro Tools compatible firewire drive enclosures so I wasn’t surprised to find a dock there that would fit the bill. The dock is the Voyager Q, and it has just about every interface you can imagine having (eSATA, Firewire 400, Firewire 800, USB 2.0) which gives us a lot of flexibility since most of our ports are full on our FOH machines.

For hard drives, I purchased 2 Seagate Barracuda 1 TB drives. We’re not going to record directly to these drives in the immediate future, but if the need arises these should fit the bill. Instead we’re going to use these drives to solve two challenges. For starters, they give us a good ole fashioned sneaker-net for transporting data. I put a dock at FOH and a dock in the studio so our studio guys can easily go grab the drive from FOH during the week and mount it on their system in the studio. The nature of the drive and dock also gives them the right spec’s for playback so they can start working immediately instead of waiting for files to transfer.

The second issue solved is keeping our recording drives free and clear. We have two drives in each FOH location with tracks spanning across the drives for maximum stability. I’ve probably recorded 64 tracks on one of our existing tracking drives for 2-3 hours straight in the past, however, that’s not a supported or endorsed track count by Avid, and now that these are the main recording drives I’m trying to minimize risks as much as possible. With these new drives, Sunday sessions will be offloaded each week so that the recording drives can be cleaned before the next Sunday.

Something I should note is we don’t currently archive our entire multi-tracks for long term storage. There are elements that get backed up in post mode on the studio side, but it’s rarely an entire session. The new drives are for short-term storage only. So here’s how the new short-term archive works. Each one of our new drives gets used to archive the Pro Tools sessions for a month. I have a series of scripts in Automator that run Sunday afternoon and evening that consolidate and copy the sessions from each auditorium over to that month’s designated drive. The scripts also grab two-track masters of the entire service in each auditorium and place them in a folder that any of our production or music staff can access during the week if they need a simple board mix. This way everything is ready to go on Monday morning. At the beginning of the next month, the drive goes in an anti-static case and gets put on the shelf for short-term storage while we pull the other drive and re-formatted it for that month’s use. The beauty of this is if we end up needing to archive things a bit longer archive, we can easily add more hard drives to put things on a 3 or 4 or 5 or 12 month rotation.

Something I would like to add, though, is if we move to permanent archives of our multi-tracks, we’ll need to find something else. Hard drives, unfortunately, aren’t the best way to store things long term. I know too many people who have made records and put the hard drive on a shelf only to go back to it a few years later to find that the drive would no longer spin-up. I feel pretty good about the Seagates we’re using since they come with a 3 year warranty, but it’s still not something I would bank on long term.

David Stagl

4 Responses to “That Boring File Management Stuff….

  • It’s unfortunate solid state drives aren’t big enough (in memory capacity) yet to really work for your application. But even then, would you really want to save the information for all time? Most archived material sits dormant collecting dust, never being used again. And the only time it gets taken out is when the format is about to go obsolete and you feel like you need to convert it to the new medium, such as analogue tape to PT transfers. But even after doing loads of those, the hard drives just went back into storage to replace the big, heavy 2″ tape reels. Instead of doing that, it sounds like you should develop a system of judging and tossing 80% of the multi-track masters that are over a year old, and keep that top 20% forever, and sticking to the decision the same way engineers had to commit decisions to tape back in analogue days. We do that with videos. Anything over a year old gets tossed, unless we feel that Sunday was really special or really defined what our church vision is about. That stuff we want to keep.

  • I think you’re on to something, JB. However, in my present role, I don’t feel it’s my position to judge what gets kept and what doesn’t. Plus, in my experience, it has always been best to save everything because typically the day you start erasing is the day before the client calls.

  • I was a little taken back by hard drives not being a good place for long term storage. We currently transfer video editing files to external drives when we are done with a project. These drives sit on a shelf just in case. If you come up with an alternate solution let me know. The dock looks awesome! We are going to have to do that instead of buying the whole enclosure every time. Thanks for the tips…

    • To my knowledge, tape is probably still the most reliable format for long-term storage; something like LTO or DLT. Optical is probably the next best for reliability–DVD or Blu-Ray. Solid state might be a good option in the future, but I don’t think it’s there, yet. The problem is like JB mentioned, formats go obsolete, and then what do you do.

      If you need to store something for a year or so, hard drives probably won’t be a problem, BUT I have seen drives that have been on a shelf not want to spin up again so it might be a good idea to keep an additional safety for really critical data. Anything I have that is absolutely critical to keep stays on a server or on a local machine is worked on and backed up regularly.