Search Results for '"truth is in the file"'

Back in February I noted that Photo Gallery can embed metadata — including tags — directly into images. One implication of this “truth in the file” strategy was that photos tagged on the desktop in Photo Gallery would someday be able to carry their tags with them when uploaded to Flickr. Today is that day. The latest beta refresh of Photo Gallery introduces a tag-preserving Flickr upload feature. Excellent!

There is, of course, no absolute truth in this game of tag. From Photo Gallery’s perspective, not all image formats are created equal. Some are friendlier than others to the embedding of metadata. So while you can tag both a JPEG and a PNG image in Photo Gallery, the tags have different relationships to the photos. In the case of the JPEG, the tag lives inside the image file. In the case of the PNG, it lives in an external data store.

It was already the case that a tag assigned to a PNG image would not survive a trip through your local recycle bin. For the same reason, a tag assigned to a PNG image will not survive a trip to Flickr.

What about the reverse trip? If you download a tagged JPEG from Flickr, will Photo Gallery see the tag? So far as I can determine, no. (Update: As per kellan’s comment below, the largest size of your photo — and the one you’re likeliest to want to fetch back — is unmodified and does retain the tag.)

It follows from these observations that there is no straightforward way to synchronize your cloud-based tag vocabulary with your local tag vocabulary. If these evolve, they will diverge.

Mind you, I’m not complaining. There really is no practical solution to this problem. I’m delighted that we’ve taken this first step, not only because it offers real practical benefit but also because it will get more people thinking about how we might want to manage our self-assigned metadata in a decentralized world.

Here’s what I envision. In a hosted lifebits scenario, every chunk of bits I produce — on the desktop or in the cloud — has a canonical, globally-unique, and cloud-accessible name, along with one or more representations and associated pieces of metadata. When I upload a photo to Flickr I don’t transfer the bits that directly represent the photo. Instead I transfer that canonical name. When Flickr or another service needs to materialize a representation of the photo, it sources the image data from one of my hosted representations. These services likewise syndicate my metadata, including tags, so that when my tag vocabulary evolves, it evolves everywhere.

I know, I know, there is a crazy amount of stuff that would need to happen in order to make this possible. But it would be good stuff, for users and for service providers, and I can’t help imagining what it would be like.

For the last five years, Bill Crow has been working on HD Photo, a new image file format that’s intended to supplant the JPEG format currently at the heart of the digital photography ecosystem.

I first met Bill many years ago when he came to BYTE to show us HP NewWave, which was probably the earliest effort to produce an object-oriented file system for Windows — originally, believe it or not, for Windows 1.0. The connection between NewWave and HD Photo is tenuous, but it does exist in the sense that the metadata strategies we’re seeing today (see the truth is in the file) point the way toward ending the tyranny of the hierarchical file system.

Today’s podcast begins by revisiting NewWave, but it’s mostly about HD Photo: Why it was created, how it works, what it will mean to both amateur photographers (“happy snappers”) as well as pros, and how it will be standardized and baked into a next generation of digital cameras.

Along the way I learned a huge amount about the current state of digital photography. For example, I knew that pros prefer to shoot in RAW format, but I wasn’t clear what that meant. According to Bill, a RAW image is just sensor data from a high-end camera, which photo processing software later turns into an image. The professional photographer trades away convenience for control and flexibility. In the case of the JPEG images produced by the vast majority of digicams, though, it’s the other way around. We get usable images without any fuss, but we give up the ability to reinterpret the data. HD Photo aims for the best of both worlds: ultimate control and flexibility if you desire, convenience when you don’t.

Although Bill guesses we’re two years away from commercial HD Photo cameras, the format is being used today to support Photosynth. As he explains on his blog, a compressed Photo HD image has a regular structure that makes it possible to extract images at various levels of detail without decoding the entire image.

There’s a whole lot more to the story. I hugely enjoyed this conversation, and I think you will too.

I’ve recently been exploring the implications of the following mantra:

The truth is in the file.

In this context it refers to a strategy for managing metadata (e.g., tags) primarily in digital files (e.g., JPEG images, Word documents) and only secondarily in a database derived from those files.

Commenting on an entry that explores how Vista uses this technique for photo tags, Brian Dorsey throws down a warning flag:

Many applications are guilty of changing JPEGs [ed: RAW file, not JPEGs, are the issue, see below] behind the scenes and there is nothing forcing them to do it in compatible ways. Here is a recent example with Vista.

A cautionary tale, indeed. This is the kind of subject that doesn’t necessarily yield right and wrong answers. But we can at least put the various options on the table and discuss them.

There is an interesting comparison to be made, for example, between OS X and Vista. While researching this topic I found this Lifehacker article on a feature of OS X that I completely missed. You can tag a file in the GetInfo dialog, and when you do, the file will be instantly findable (by that tag) in SpotLight.

My purpose here is not to discuss or debate the OS X and Vista interfaces for tagging files and searching for tagged files. I do however want to explore the implications of two different strategies: “the truth is in the file” versus “the truth is in the database”.

In Vista, if I tag yellowflower.jpg with iris, that tag lives primarily in the file yellowflower.jpg and secondarily in a database. An advantage is that if I transfer that file to another operating system, or to a cloud-based service like Flickr, the effort I’ve invested in tagging that file is (or anyway can be) preserved. A disadvantage, as Brian points out, is that when different applications try to manage data that’s living inside JPEG files, my investment in tagging can be lost.

Conversely, if I tag yellowflower.jpg with iris in OS X, yellowflower.jpg is untouched, the tag only lives in Spotlight’s database. If I transfer the file elsewhere, my investment in tagging is lost. But on my own system, my tags are less vulnerable to corruption.

Arguably these are both valid strategies. The Vista way optimizes for cross-system interoperability and collaboration, while the OS X way optimizes for single-system consistency. Of course as always we’d really like to have the best of both worlds. Can we?

It’s a tough problem. Vista tries to help with consistency by offering APIs in the .NET Framework for manipulating photo metadata. But those APIs don’t yet cover all the image formats, and even if they did, there’s nothing to prevent developers from going around them and writing straight to the files.

For its part, OS X offers APIs for querying the Spotlight database. So an application that wanted to marry up images and their metadata could do so, but there’s no guarantee that a backup application or a Flickr uploader would do so.

It’s an interesting conundrum. Because I am mindful of the lively discussion over at Scoble’s place about what matters to people in the real world, though, I don’t want to leave this in the realm of technical arcana. There are real risks and benefits associated with each of these strategies. And while it’s true that people want things to Just Work, that means different things to different people.

If you’re an avid Flickr user, if you invest effort tagging photos in OS X, and if that effort is lost when you upload to Flickr, then OS X did not Just Work for you. Conversely if you don’t care about online photo sharing, if you invest effort tagging photos in Vista, and then another application corrupts your tags, then Vista did not Just Work for you.

I think many people would understand that explanation. In principle, both operating systems could frame the issue in exactly those terms, and could even offer a choice of strategy based on your preferred workstyle. In practice that’s problematic because people don’t really want choice, they want things to Just Work, and they’d like technology to divine what Just Work means to them, which it can’t. It’s also problematic because framing the choice requires a frank assessment of both risks and benefits, and no vendor wants to talk about risks.

I guess that in the end, both systems are going to have to bite the bullet and figure out how to Just Work for everybody.

In 2005 I noted the following two definitions of truth:

1. WinFS architect Quentin Clark: “We [i.e. the WinFS database] are the truth.”

2. Indigo architect Don Box: “Message, oh Message / The Truth Is On The Wire / There Is Nothing Else”

Today I’m adding a third definition:

3. Scott Dart, program manager for the Vista Photo Gallery: “The truth is in the file.”

What Scott means is that although image metadata is cached in a database, so that Photo Gallery can search and organize quickly, the canonical location for metadata, including tags, is the file itself. As a result, when you use Photo Gallery to tag your images, you’re making an investment in the image files themselves. If you copy those files to another machine, or upload them to the Net, the tags will travel with those image files. Other applications will be able to make them visible and editable, and those edits can flow back to your local store if you transfer the files back.

That’s huge. It’s also, of course, a bit more complicated. As Scott explains, there are different flavors of metadata: EXIF, IPTC, and the new favorite, XMP. And not all image formats can embed image metadata. In fact many popular formats can’t, including PNG, GIF, and BMP. [Update: Incorrect, see next rock.] But JPG can, and it’s a wonderful thing to behold.

For example, I selected a picture of a yellow flower in Photo Gallery and tagged it with flower. Here’s the XML that showed up inside yellowflower.jpg:

<xmp:xmpmeta xmlns:xmp="adobe:ns:meta/">
<rdf:RDF xmlns:rdf="">
<rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" 
<rdf:Bag xmlns:rdf="">
<rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" 
  <rdf:Bag xmlns:rdf="">
<rdf:Description xmlns:MicrosoftPhoto="">
<rdf:Description xmlns:xmp="">

It’s a bit of a mish-mash, to say the least. There’s RDF (Resource Description Framework) syntax, Adobe-style metadata syntax, and Microsoft-style metadata syntax. But it works. And when I look at this it strikes me that here, finally, is a microformat that has a shot at reaching critical mass.

Perhaps we’ve been looking in the wrong places for the first microformat to achieve liftoff. Many of us hoped hCalendar would, but it’s hard to argue that it has. I suppose that’s partly because even though we have a variety of online event services that produce the hCalendar format, there just aren’t that many people publishing and annotating that many events.

There are already a lot of people saving, publishing, and annotating photos. And the tagging interface in Vista’s Photo Gallery, which is really sweet, is about to recruit a whole lot more.

There’s also good support in .NET Framework 3.0 for reading and writing XMP metadata. In the example above, the tag flower was assigned interactively in Photo Gallery. Here’s an IronPython script to read that tag, and change it to iris.

import clr
from System.IO import FileStream, FileMode, FileAccess, FileShare
from System.Windows.Media.Imaging import JpegBitmapDecoder, 

def ReadFirstTag(jpg):
  f = FileStream(jpg,FileMode.Open)
  decoder = JpegBitmapDecoder(f, BitmapCreateOptions.PreservePixelFormat, 
  frame = decoder.Frames[0]
  metadata = frame.Metadata
  return metadata.GetQuery("/xmp/dc:subject/{int=0}")

def WriteFirstTag(jpg,tag):
  f = FileStream(jpg,FileMode.Open, FileAccess.ReadWrite, 
  decoder = JpegBitmapDecoder(f, BitmapCreateOptions.PreservePixelFormat, 
  frame = decoder.Frames[0]
  writer = frame.CreateInPlaceBitmapMetadataWriter()
    print "cannot save metadata"

print ReadFirstTag('yellowflower.jpg') 
print ReadFirstTag('yellowflower.jpg')

The output of this script is:


And when you revisit the photo in Photo Gallery, the tag has indeed changed from flower to iris. Very cool.


Get every new post delivered to your Inbox.

Join 6,064 other followers