Scalability in Photography
I’ve always enjoyed thinking about scalability. Way back when, I used to be a database programmer (I still do some, but mostly just for fun). In the world of databases, scalability is a big issue. How much data can you get into a database before things just fall apart, with performance so bad you might as well not have the data at all? With very basic database tools, you can’t put too much data in before things get difficult. So when planning a database you had to think of the tools and technology you were using in the context of how much data would eventually accumulate in the database.
I know you’re probably wondering by now what this has to do with photography…
A couple weeks ago I had the pleasure of meeting with the Vancouver Organizing Committee for the 2010 Olympic Games in Vancouver, Canada. This got me thinking about scalability in photography. A couple weeks of many events per day, hundreds of photographers, and image counts that boggle the mind. In talking to a photographer from Sports Illustrated I learned they typically capture 25,000 per day at the Olympics.
(Which brings me back to scalability.)
So, is photography scalable? Digital has removed many barriers to capturing huge numbers of images, but it hasn’t really (at least so far) provided exceptional tools for actually dealing with that volume of images. What’s a photographer to do? How does sorting and organizing your images possibly keep up with the number of images the typical photographer captures?
I’d be curious to learn what photographers are doing to manage this growing challenge…
Explore posts in the same categories: Photography, Digital
November 15th, 2007 at 6:30 am
I’m having a massive problem with this as I take more and more photos. I have a very good workflow that has worked well for me in the past, basically involving a clean year, month, day based hierarchy of folders in Windows Explorer. I tag every photo with metadata for future searching, but I have no clean and easy way to actually do that searching in a timely manner. I really don’t want to have to switch to a database type tool such as Lightroom because I don’t want to have to create new catalogs (or libraries or whatever they call them) but I’m starting to get buried under the volume of photos. I’m definitely looking for something new, and I don’t even come close to the volume of photos you are talking about…
November 15th, 2007 at 7:49 am
I’m optimistically hoping technology is going to help with this. I’m already geotagging my photos so I can search by location automatically. I’m hoping face recognition software will let me find photos of people I’ve trained it on. Also, we need some tools to help identify “good” photos. I’m thinking after the fact focus detection software, or sharpness detection. My problem is that database software like lightroom and portfolio and so on, gets really slow when you have 100,000 images in them. So there’s some basic performance improvements needed as well (without having to pay for $10k pieces of software)
In the end you don’t really need 25,000 shots. There are still going to be a defined number of shots you are going to use for mainstream (newspaper articles, magazine shots) and the rest you generate in the process of getting those “good ones”. But there is an additional “long tail” of photos, that if they are searchable open up new opportunities. Such as the work being done by researchers to recreate 3d models of landmarks from millions of snapshots taken by tourists from millions of different angles. Or sometime in the future an olympic athlete that wasn’t a superstar contender , wins a nobel peace prize, you may not know you need her photo but later you’ll really wish for it. Of course that’s only useful if you can go back and find that photo!
November 15th, 2007 at 7:54 am
By the way, I forgot to mention the same problem applies in the scientific/medical industry. The explosion of medical imaging, and new microscopy techniques is generating tons of images (in 3d) that without cataloging, are useless. We’re building image databases designed to scale to the petabyte level from a storage perspective, but as you point out, without some new way of making use of that data, it’s just going to be there using up disk space being saved for liability reasons rather than pushing forward the frontiers of science.
We can only do some much with drag and drop tagging in web pages and other usability improvements when considering hundreds of thousands of photos.
To some extent you can tackle some of these issues by opening up your data putting it on the web and encouraging lots of friends and family (or collaborator scientists) to tag the data they are specifically interested in (a la flickr or facebook) The trick there is having a common taxonomy, and actually motivating people to do it.
November 16th, 2007 at 6:49 am
Not only is the issue the number of Photos, but the size of the image files. I have several images I scanned from medium format film, applied some layers and they are over 700MB in size. I use Microsofts Expression Media to catalog the images and the software won’t generate a displayable thumb nail on these images. One positive note, if I have all my metadata added in Bridge/Photoshop and my catalog get destroyed, I just reimport my folders and all my metadata comes back!
December 4th, 2007 at 5:21 am
I have heard so many times about the use of iView Media Pro from professional photographers but the cost seems always to cause one to ponder the real vs supposed benefits. Scalability is for me becoming a problem as I have, like so many added HDD’s both internal & external. Can a single source software be the answer? I think even large stock photography agencies are still struggling with this. Answers in the form of hard/software solutions are always just around corner.
Personally with some 14k image files I am finding that Bridge is not very efficient at keeping them in their proper places as well as some sidecar files. But I must admit to not being the most adept at ALWAYS doing the right things when I save. I will defer to those whom are currently having success with their applications & would welcome input in advising which are the best.
Thanks Tim for a great subject to discuss.