Truth, files, microformats, and XMP

In 2005 I noted the following two definitions of truth:

1. WinFS architect Quentin Clark: “We [i.e. the WinFS database] are the truth.”

2. Indigo architect Don Box: “Message, oh Message / The Truth Is On The Wire / There Is Nothing Else”

Today I’m adding a third definition:

3. Scott Dart, program manager for the Vista Photo Gallery: “The truth is in the file.”

What Scott means is that although image metadata is cached in a database, so that Photo Gallery can search and organize quickly, the canonical location for metadata, including tags, is the file itself. As a result, when you use Photo Gallery to tag your images, you’re making an investment in the image files themselves. If you copy those files to another machine, or upload them to the Net, the tags will travel with those image files. Other applications will be able to make them visible and editable, and those edits can flow back to your local store if you transfer the files back.

That’s huge. It’s also, of course, a bit more complicated. As Scott explains, there are different flavors of metadata: EXIF, IPTC, and the new favorite, XMP. And not all image formats can embed image metadata. In fact many popular formats can’t, including PNG, GIF, and BMP. [Update: Incorrect, see next rock.] But JPG can, and it’s a wonderful thing to behold.

For example, I selected a picture of a yellow flower in Photo Gallery and tagged it with flower. Here’s the XML that showed up inside yellowflower.jpg:

<xmp:xmpmeta xmlns:xmp="adobe:ns:meta/">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" 
  xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:subject>
<rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:li>horse</rdf:li></rdf:Bag>
</dc:subject>
</rdf:Description>
<rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" 
  xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0">
  <MicrosoftPhoto:LastKeywordXMP>
  <rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:li>flower</rdf:li>
  </rdf:Bag>
  </MicrosoftPhoto:LastKeywordXMP>
</rdf:Description>
<rdf:Description xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0">
  <MicrosoftPhoto:Rating>1</MicrosoftPhoto:Rating>
 </rdf:Description>
<rdf:Description xmlns:xmp="http://ns.adobe.com/xap/1.0/">
  <xmp:Rating>1</xmp:Rating>
</rdf:Description>
</rdf:RDF>
</xmp:xmpmeta>

It’s a bit of a mish-mash, to say the least. There’s RDF (Resource Description Framework) syntax, Adobe-style metadata syntax, and Microsoft-style metadata syntax. But it works. And when I look at this it strikes me that here, finally, is a microformat that has a shot at reaching critical mass.

Perhaps we’ve been looking in the wrong places for the first microformat to achieve liftoff. Many of us hoped hCalendar would, but it’s hard to argue that it has. I suppose that’s partly because even though we have a variety of online event services that produce the hCalendar format, there just aren’t that many people publishing and annotating that many events.

There are already a lot of people saving, publishing, and annotating photos. And the tagging interface in Vista’s Photo Gallery, which is really sweet, is about to recruit a whole lot more.

There’s also good support in .NET Framework 3.0 for reading and writing XMP metadata. In the example above, the tag flower was assigned interactively in Photo Gallery. Here’s an IronPython script to read that tag, and change it to iris.

import clr
clr.AddReferenceByPartialName("PresentationCore")
from System.IO import FileStream, FileMode, FileAccess, FileShare
from System.Windows.Media.Imaging import JpegBitmapDecoder, 
 BitmapCreateOptions,BitmapCacheOption


def ReadFirstTag(jpg):
  f = FileStream(jpg,FileMode.Open)
  decoder = JpegBitmapDecoder(f, BitmapCreateOptions.PreservePixelFormat, 
    BitmapCacheOption.Default)
  frame = decoder.Frames[0]
  metadata = frame.Metadata
  f.Close()
  return metadata.GetQuery("/xmp/dc:subject/{int=0}")


def WriteFirstTag(jpg,tag):
  f = FileStream(jpg,FileMode.Open, FileAccess.ReadWrite, 
    FileShare.ReadWrite)
  decoder = JpegBitmapDecoder(f, BitmapCreateOptions.PreservePixelFormat, 
    BitmapCacheOption.Default)
  frame = decoder.Frames[0]
  writer = frame.CreateInPlaceBitmapMetadataWriter()
  try:
    writer.SetQuery("/xmp/dc:subject/{int=0}",tag)
    writer.TrySave()
  except:
    print "cannot save metadata"
  f.Close()
  writer.GetQuery("/xmp/dc:subject/{int=0}")

print ReadFirstTag('yellowflower.jpg') 
WriteFirstTag('yellowflower.jpg','iris')
print ReadFirstTag('yellowflower.jpg')

The output of this script is:

flower
iris

And when you revisit the photo in Photo Gallery, the tag has indeed changed from flower to iris. Very cool.

47 thoughts on “Truth, files, microformats, and XMP

  1. Steven Johnson

    PNG can embed prettty much anything it wants (though it may not be common to embed metadata in PNG).

    Adobe Fireworks embeds its entire source file as metadata in PNG files!

    Reply
  2. Pingback: Mike Linksvayer » Perils of a too cool name

  3. Dan Connolly

    Great to see RDF support in Photo Gallery.

    Can you tweak the python script to not use “import *”? I’d like to know which objects come from which import.

    And yeah, PNG can embed metadata just fine. Please take it out of the list of formats that can’t.

    Reply
  4. Stuart Weibel

    There is much of value in this post, in particular the benefit of embedding metadata in image files.

    I believe characterizing this as a possible first microformat “killer ap” overreaches the bounds of microformats, and underestimates the difficulties in integrating multi source metadata.

    I’ve elaborated on these themes in a post of my own:
    http://weibel-lines.typepad.com/weibelines/2007/02/jon_udell_who_i.html

    keep up the good work.

    stu

    Reply
  5. Pingback: Elias Torres » Blog Archive » XMP and microformats == RDFa

  6. Jon Udell Post author

    “Can you tweak the python script to not use “import *”?”

    Done. Good suggestion, that’s a bad habit.

    “And yeah, PNG can embed metadata just fine. Please take it out of the list of formats that can’t.”

    Done.

    Reply
  7. Bruce D'Arcus

    I mentioned this on Elias’ response to this post (in trackback), but just to be clear, XMP is just an RDF subset; as Erik mentions, a rather bizarre one that actually throws out much of what is more useful about RDF (reference via URI).

    This has nothing to do with microformats.

    Reply
  8. Pingback: All in a days work…

  9. Mark Birbeck

    MICROFORMATS AND RDFA

    You say:

    “Perhaps we’ve been looking in the wrong places for the first
    microformat to achieve liftoff. Many of us hoped hCalendar
    would, but it’s hard to argue that it has. I suppose that’s
    partly because even though we have a variety of online event
    services that produce the hCalendar format, there just aren’t
    that many people publishing and annotating that many events.”

    I think this misses the point; the problem with hCalendar is not that there isn’t enough data…there’s plenty! The problem lies in the general lack of coherence of microformats. You can’t take a collection of non-integrated things and call them a standard simply by giving them a common moniker. But that unfortunately is what microformats has done (and makes a virtue of it!).

    The result of doing this, is that this week you have to write an hCalendar parser, next week you have to write an hCard parser, the week after you haven’t got time to write a parser because you are too busy trying to convert FOAF to hFOAF, and then the following week you have to define how all the microformats you have so far are to work together, just in time to start all over again with the next format (which incidentally involves you in learning about chemistry). In short, it’s a technology that does not scale.

    RDFa on the other hand, has confronted the issue of creating a *generic* syntax for putting metadata into HTML right from the start, because that was it’s whole _raison d’etre_. Once you have written one RDFa parser, you need write no more, unless of course you want another for a different platform. And you don’t need to go around converting RDF vocabularies as microformats does, since RDF is used directly. (Note that I’m specifically saying that *RDF* is used, and not *RDF/XML*. RDFa uses the idea of representing metadata with triples–such as [this document] [written by] [Jon Udell] or [this document] [published on] [Thursday], etc.–but it doesn’t use the tricky and complicated RDF/XML syntax to represent those triples.)

    The whole point is that by providing _general_ rules, any vocabulary–available now, or coming in the future–can be used directly in an HTML document.

    So whilst microformats don’t scale, RDFa absolutely does. And crucially, by being able to use and mix already existing vocabularies without conversion, RDFa retains the de-centred model that the web has thrived on.

    MASH-UPS

    And this de-centred model is also what mash-ups are all about; the mash-up lift-off you’ve waited for hasn’t happened because in order to do even a simple mash-up with calendar and people info that is based on microformats, we have to write two parsers, and then work out how the two sets of data co-exist on the page. In short, we’re trying to work with inherently de-centred mash-ups, using the highly centralised microformats.

    RDFa returns us to the decentralise nature of HTML (anyone can link to anything) and the decentralised nature of RDF (anyone can make statements about anything) and leverages the two–“bridging the clickable and semantic webs”.

    BLATANT PLUG

    Anyone who is in London next week might find the following meeting on the Semantic Web of interest:

    There will be four speakers including me, and I’ll be using my spot to explain RDFa, and do some demos.

    And the blog-post just linked to contains some RDFa that describes the meeting… ;)

    PS…ON THE UNIVERSAL DOCUMENT

    I’ve probably used up too much of your valuable server-space already, but if I can add a postscript in agreement with the general idea of having one metadata structure that can be used in many document types, but to suggest an alternative to XMP. A couple of years ago I proposes a universal document format that would provide a standard document/metadata structure:

    The idea is to make XHTML a host language; it has a ‘head’ that contains metadata, and a ‘body’ that could easily contain any type of ‘document’. I discuss what I mean by a ‘document’ as follows:

    “By a document I mean pretty much anything that you would commonly
    refer to as a document — a spreadsheet, a memo, a chapter of your
    autobiography, a piece of music, a blog entry, even a vector graphics
    image. However, I also mean pretty much anything that you would not
    commonly refer to as a document — a list of RSS feeds, a list of
    RSS articles, a friend-of-a-friend (FoaF) file, even a configuration
    file for your email software.”

    The post looks at using this technique for email, OPML file, configuration files, and so on.

    Reply
  10. Alexander Pohoyda

    Why is this RDF snippet so suboptimal? What’s the reason of declaring the “rdf” namespace 3 times and the “MicrosoftPhoto” namespace 2 times? The “xmp” namespace is also declared twice, but with two different URIs!

    Without the “rdf:about” attribute, both rating declarations are absolutely useless, as far as I can tell.
    And, by the way, you only need one “rdf:Description” tag in this example.

    I’d suggest to use the N3 format instead:

    @prefix rdf: .
    @prefix dc: .
    @prefix MicrosoftPhoto: .
    @prefix xmp: .
    @prefix uuid: .

    uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b
    dc:subject [ a rdf:Bag; rdf:li "horse" ];
    MicrosoftPhoto:LastKeywordXMP [ a rdf:Bag; rdf:li "flower" ];
    MicrosoftPhoto:Rating 1;
    xmp:Rating 1.

    However, the worst mistake in this example is that both “horse” and “flower” tags are used as a value, instead of as a resource.
    Why is this bad? Because the RDF is a Resource Description Framweork, not a string description framework.

    It seems that you have thrown away the baby and left the water (RDF/XML).

    Reply
  11. Pingback: Blogging from Word 2007, crossing the chasm « Jon Udell

  12. Brian Dorsey

    John,

    I always enjoy reading your take on the problems we face. However, in this instance I completely disagree with where the truth should be stored, at least in the case of digital photographs. As you mention, there is already a lot of metadata stored in the files themselves. The problem I’ve had in the past with allowing ANY application to modify my files directly is that they all interpret the various standards differently and end up losing some of the information.

    The specific example which got me looking into this problem in detail was when my EXIF data disappeared after doing a lossless JPG rotation. I’ve since run into it in many places, some applications like to edit IPTC data directly in the file, etc. I now move my images from my memory card directly to a read only file share and browse them from there to prevent applications from editing my files.

    Many applications are guilty of changing JPEGs behind the scenes and there is nothing forcing them to do it in compatible ways. Here is a recent example with Vista:
    http://lifehacker.com/software/vista/vista-mangles-metadata-235265.php

    Take care,
    -Brian

    Reply
  13. Pingback: Who’s got the tag? Database truth versus file truth, part 3 « Jon Udell

  14. trevor

    hi, i am somewhat frustrated by your remark at # 15… “so where should the truth reside?”

    because “where” misleads, and implies that some “distance” between the truth and the object could neccessarily provide a totally “safe” place for that truth.

    even if the “truth” was somewhere else, it could still be abused and corrupted. just because currently it rides piggyback with the object data, does not imply that they are ONE.

    they already are seperate entities, and are modified as such.

    putting more distance between the two does not strengthen the bond, or the safety of either piece – but they are seperate.

    i agree solidly with mark birbeck, comment 10.

    a universally accepted header (wrapper, whatever) that keeps a tight bond to the file, and is extensible makes sense – because the two already ARE seperately houseable. but they need each other to have meaning.

    if you split them apart and send them each inseperate directions, you would have 4 new distinct entities then, each that should self describe (with a header), and a content

    – like cell division.

    Reply
  15. Mary Branscombe

    There’s another problem with microformats. The data in them in is canonised as eternally true, but calendar events change. For mIX 06 Microsoft published the agenda as microformat information that I could open and save into my Outlook calendar. Then the entire agenda changed and I could either delete everything and open them all again (tricky when I’d been picking one out of four events for each slot) or edit it all by hand. I’ve hated microformats with a passion since. I don’t know if adding a UID or a ‘last updated’ field to the microformat would be enough to solve the problem?

    my vote for microformat most likely to succeed? a geotag. I want events, car rental agreements, hotel and plane bookings to all have a geotag attached.

    Reply
  16. Jon Udell Post author

    “For mIX 06 Microsoft published the agenda as microformat information that I could open and save into my Outlook calendar.”

    Yeah, that’s a tough tradeoff. I assume it was possible to also subscribe wholesale to the calendar, in which case you’d have had live refresh, but it would have been information overload?

    In principle there could also be, and should also be, live refresh on a per-item basis. I recall correctly that was part of the LiveClipboard demo at ETech last year. But it may not yet have been used in the wild.

    Reply
  17. Alex James

    The truth should be in the File *and* the Database.

    The key to this is tweaking WinFS’ idea of promotion and demotion so that it is aware of the embedded general purpose meta-data i.e. the XMP or RDF or whatever.

    Reply
  18. Max

    I’ve found this simple WSH script to retrieve some information stored in jpg photo (exif attributes):

    Dim arrHeaders(34)
    Set objShell = CreateObject(“Shell.Application”)
    Set objFolder = objShell.Namespace(“D:\_x”)
    For i = 0 to 33
    arrHeaders(i) = objFolder.GetDetailsOf(objFolder.Items, i)
    Next
    For Each strFileName in objFolder.Items
    For i = 0 to 33
    Wscript.Echo i & vbtab & arrHeaders(i) _
    & “: ” & objFolder.GetDetailsOf(strFileName, i)
    Next
    Next

    It works fine in windows XP!
    Is it possible to read and modify new XMP tag via WSH in Vista??? Could you tell me how?

    Thanks in advance.
    Max

    Reply
  19. Pingback: Bob On Development » The Dark Side of Tagging in Windows Vista

  20. Pingback:

  21. Pingback:

  22. Pingback:

  23. Pingback: www 8313 ru

  24. Pingback:

  25. Pingback: “Truth is in the file” redux: From Photo Gallery to Flickr « Jon Udell

  26. Voip

    Hi, my name is Chris Voip. I always enjoy reading your take on the problems we face. However, in this instance I completely disagree with where the truth should be stored, at least in the case of digital photographs. As you mention, there is already a lot of metadata stored in the files themselves. The problem I’ve had in the past with allowing ANY application to modify my files directly is that they all interpret the various standards differently and end up losing some of the information. Chris

    Reply
  27. Hans Fremuth

    An open standard such as XMP is like a knife. You can use it to spread butter or stab someone to death. The key is discipline in software development. Don’t “stab” your users by implementing new XMP namespaces or elements, unless you know what you are doing and (equally important) what others do. This applies also to the follow developers at MS: I cannot find a serious reason why they implemented a separate rating element in a custom XMP namespace, it already existed. Same Faux Pas for nested (“hierarchical”) keywords: MS Expression Media and Adobe Lightroom use the pipe symbol as separators, as suggested by many subject experts – the MS Photo Gallery team used slash as a separator, not smart (… and not needed).

    Reply
  28. Pingback: John Federico » Blog Archive » Windows Live Photo Gallery

  29. Universal Removals

    Including Changing Tables, oxidize if not?Migrant business owner, stocks or from.For any site, profit web site.Rental as these Universal Removals, move yet to payment Dont promise.To Walker, gifts and business.,

    Reply
  30. Pingback: WebFS: a Web of Data - Zef.me

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s