Wakened this morning, about three o’clock, by Mr. Griffin with a letter from Sir W. Coventry to W. Pen…
So begins today’s installment of The Diary of Samuel Pepys, as rendered by Phil Gyford. It’s a remarkable project that maps January 1, 1660 (the start of Pepys’ famous diary) to January 1, 2003 (the start of Phil’s Moveable Type recreation of the diary) and has continued faithfully ever since.
The Pepys blog is enhanced in all sorts of useful ways. People, places, and topics are cross-linked with indexes, places are mapped, all references are viewable on a timeline — it’s a brilliant example of advanced blog customization.
Back in 2003 I mused about what kind of content management system would enable somebody to do a project like this without a lot of inspired hacking. The question came up again recently when my sister Ruth decided to recreate an archive of letters that my parents wrote home from our 15-month stay in New Delhi during 1961 and 1962.
I’ve long held that blog publishing systems are really lightweight content management systems that can be used for almost any purpose. So I pointed her to WordPress.com, explained that you can use pages instead of posts to arrange items however you like, and waited to see what would happen.
Well, it didn’t work. It’s true that you can build an arbitrary collection of pages, but there’s no way Ruth would be able to manage that collection without automation. I could write code to help her, but I don’t want to. That’s partly laziness, and partly curiosity about how to use the standard kit to achieve the desired effects.
One of the biggest limitations of pages, in WordPress, is something I’d never noticed until now: No tags! So ended my plan to have Ruth use tags on pages to achieve a lightweight version of Phil Gyford’s indexes.
Why not just use posts? Originally I thought it would be cool to mimic the Pepys diary: start with a date in 1961, and continue in “real” time. But Ruth doesn’t want to do it that way. She wants to be able to process the archive in any order that’s convenient. And she wants it to read forward, like a book of letters, not backward like a blog. These perfectly reasonable requirements turn out to be harder to satisfy than you’d think.
It turns out that you can make the letters run forward on the Posts page by manipulating the publication dates. So here was the scheme I tried first:
July 2 1961 -> Jan 01 1961 15:01 July 4 1961 -> Jan 01 1961 15:00 ... Oct 19 1961 -> Jan 01 1961 04:01 Oct 22 1961 -> Jan 01 1961 04:00
In this scheme, every letter maps to the same day, chosen arbitrarily as Jan 1, 1961. Every month maps to an hour of that day, each letter maps to a minute within that hour, and the times run backward. Since WordPress reverses the sequence again when displaying items on the Posts page, that makes time run forward in that view.
The benefits are huge. Now Ruth can use tags to organize sets of letters, imposing as much or as little structure as she wants. Views by tag are neatly presented as sets of blurbs with “Continue reading” links. Each item automatically links to its predecessor and successor.
But there’s irreducible weirdness too. For example, the Jan 01 1961 date — which has now become an abstract database key used only for sorting — is part of every post URL. You wind up with patterns like this:
/1961/01/01/june-30-1961-from-anita/
This gets even weirder because dates prior to the start of Unix time — Jan 1, 1970 — don’t display in the management UI. However that turns out to be both a feature and a bug. It’s a feature because WordPress reverts to the current date for display, so you see “Posted on June 28, 2010 by Ruth” instead of “Posted on January 1, 1961 by Ruth.” And it’s a bug because you can’t easily scan and adjust the dates that control sorting.
More weirdness arises from the deeply hardwired assumption — in WordPress, but also in all blogs, really — that entries post in reverse chronological order. Although the backwards time mapping seemed at first glance to work, it turned out to be broken in two ways. On the Posts page, after the break, the link pointed to “Older entries” which were really, in our scheme, “Newer entries.” And within posts, the next and previous logic was also reversed.
So for now I’ve gone back to a forward mapping of hours and minutes within Jan 1, 1961. I’ve ditched the default Posts page in favor of a hand-crafted page that presents items in ascending order. Once you’re in an item, the next and previous links work as expected because, when you move from item to item, WordPress uses a forward arrow of time.
I’m not complaining. It’s astonishing that WordPress provides a free service that Ruth can use publish this archive of letters, and I’m hugely grateful. I think we’ll be able to come up with a technique that will satisfy her requirements — without demanding heroic effort from her or custom software from me. But it sure is interesting to see what happens when you mess with a blog’s notion of the direction of time.
Hi Jon,
There seems to be a plugin to allow tagging of pages:
http://www.michelem.org/wordpress-plugin-tags4page/
and … there is a new feature in WP3.0 … “custom post type” which I think gives lots of new flexibility here.
But regardless, very interested in your approach here, due to this, something that I am currently exploring:
http://toc.oreilly.com/2010/05/-wordpress-as-book-publishing.html
there seems to be a plugin to allow tagging of pages
Yes, so I should have explained. I am embracing the cloud constraint. I want the solution to work for somebody like my sister who shouldn’t have to forgo the benefits of the fully-hosted-and-managed WordPress.com.
Re: http://toc.oreilly.com/2010/05/-wordpress-as-book-publishing.html
Interesting! I has occurred to me that this kind of thing could be a business opportunity for WordPress itself. I pay $10 a year, or whatever it is, to use custom CSS on my WP blog. My sister would pay more than that for a reliable and easy-to-use solution to her problem. And there are lots of folks like her with family archives they’d like to publish easily and effectively.
Another great online diary: George Orwell’s.
June 28, 1940’s entry is published on June 28, 2010. Fascinating to follow Orwell’s reflections on day-to-day events in wartime Britain.
http://orwelldiaries.wordpress.com/
Nice!
You can see from looking at the dates on the Archives that this project wrestled with similar issues.
The months go like this:
July 1938
August 1938
August 2008
September 2008
Interesting to see the issues you’ve wrestled with. Although Movable Type works well with very old dates, it did require some slight hacking on my part to make things work well when newer dates were expected.
For example, I want to give each entry of the diary the correct 17th century date, but I also want the system to publish the entries automatically on the correct 20th century day. So I had to add something to the MT code for that specific blog to add 343 years to each entry when working out what to automatically publish.
I’ve been meaning to write up what I’ve done on that front for years; I’ll try and do so this week.
http://drupalgardens.com would meet your cloud requirements and has a little bit more control over things like this.
Thanks for the tip, Boris.
I wonder how much out-of-the box automation you get for item-by-item navigation, viewing of tagged sets, etc?
So I had to add something to the MT code for that specific blog to add 343 years to each entry when working out what to automatically publish.
A labor of love, indeed!
Your project is an inspiration on many levels, Phil. Not least being that you’ve remained faithful to it all these years.
For this project, I’m trying to explore two related things.
1. How far can a non-technical person get, with a standard blog kit but a non-standard idea about how to use it?
2. How might the standard kit make such non-standard uses a) possible, and b) discoverable, without c) wrecking the out-of-the-box ease of use for the majority user?
As the sister referenced in this article, thanks to all for your suggestions! (And thanks to my brother for his help, although paragraph 5 did make me laugh.) I realize after reading these comments that this really is a book publishing issue, and it does seem that it is an excellent business opportunity for WordPress et al.
FYI for those offering suggestions, we also have an archive of photos (slides, which are now digitized) that we will want to incorporate somehow … preferably within the text when it relates to a specific event or person mentioned in the letter, but also with the possibility of searching the photos by tags/categories, too. Jon has suggested a link to the pro version of Flickr, and organizing/categorizing the photos there. We haven’t gotten to that point yet, but suggestions on that front (especially as to the best way to incorporate the photos into the letters) would also be welcome.
Sounds like a job for Tinderbox! (OK, it’s not the cloud, but it can publish to a static site in the cloud.)
The principle of keystroke conservation requires me to add this follow-on here instead of in email to my sister:
Note that since we’re running the dates forward again, you can use actual dates like June 2, 1961. True, they won’t show up in the WordPress admin interface. But that won’t matter, because it’ll be easy to keep things in order. And, importantly, the dates will show up in the URLs. Right now the pattern is like so:
/1961/01/01/june-22-1961-from-gene/
That 1961/01/01 is completely bogus. It was only a hack to accommodate reverse ordering. Now you can do this:
/1961/06/22/june-22-1961-from-gene/
But then why repeat the date? The URL could, and should, tell you more about each letter. Since it already contains the date, you can take that out of the slug and provide a more descriptive title. For example:
/1961/06/22/hong-kong-to-new-delhi/
Or:
/1961/06/22/from-gene-hong-kong-to-new-delhi/
Imagine reading a list of all the URLs from the letters series if they were constructed that way. The list will itself be a basic table of contents that can be easily sorted by date, and that concisely describes each letter. It could usefully be presented on an index page as a set of links, with no effort beyond writing titles for each entry.
But that’s just where you want to invest effort. The title that corresponds to the WordPress slug from-gene-hong-kong-to-new-delhi would be “From Gene: Hong Kong to New Delhi.” It will be really useful to write a title like that for each letter. Those titles will show up in a variety of contexts, ranging from the URLs to the individual letters pages to various indexes you might construct to the search results pages that people will be landing on when they discover the archive that way.
This is all just good old-fashioned information packaging — what’s called heads, decks, and leads in the journalism biz. The trick with any content management system is to do as much of this kind of packaging as the constraints of the system will allow.
Hey Boris!
How I would do this:
Back date the letters with the date they were actually sent on, so you have data fidelity.
A cool feature of WP is that you can add ?order=asc to the homepage and it will put all your entries in chronological order, and you can page through them just like you would a book.
What I’d do personally is use the archives shortcode on a page and the “page as homepage” feature to put a chronological list of all the posts right on the homepage, so people could just dive right in.
Tag the heck out of the posts
As for the dates before epoch issue, unfortunately that’s a bug here on WP.com. It’s a bit of a bizarre one, that we can only fix by hardcoding your blog to a single datacenter. Iif you contact support we’re happy to make the necessary adjustments free of charge.
This could be a job for VJOURNAL components in an iCalendar file; except that I’m not sure what “clients” (including browsers) really pay attention to this often-overlooked part of the spec.
Hmm – maybe Google Calendar would work (I really haven’t played with it) – but you could add entries with date easily enough.
Back date the letters with the date they were actually sent on, so you have data fidelity.
Yep, that’s my conclusion too.
A cool feature of WP is that you can add ?order=asc to the homepage and it will put all your entries in chronological order, and you can page through them just like you would a book.
Excellent, thanks!
Trying that on a few different blogs reveals a subtle best practice for templates. Some use Next Page and Previous Page to navigate among clumps, which is fine for any ordering.
But some use Older Entries and Newer Entries, and that breaks if you reverse the order.
What I’d do personally is use the archives shortcode on a page and the “page as homepage” feature to put a chronological list of all the posts right on the homepage, so people could just dive right in.
What Ruth wants to do is have a Home page that introduces the archive, and a Letters page where instead of a table of contents you just start reading the first letter. So the solution for now is to use “page as homepage” for Home, and a copy of the first letter for Letters. When you get to the end of that copy of the first letter, it links to the second Post in the Posts series. From there you’re off and running with forward or backward letter-to-letter navigation, plus tags for random access. I think it’ll work out really well.
Separately we’ll add an Index page and use the [archives] feature as you’ve suggested.
If you contact support we’re happy to make the necessary adjustments free of charge.
Nice. Once the dust settles we just might do that.
Thanks Matt!
Custom CSS are a premium option
http://en.wordpress.com/products/
as i said in the comment section
of the blog entry by hugh mcguire
which hugh pointed to up above,
i think that a purpose-built system
can meet the overall objective better.
so i went off and programmed one…
if ruth would be interested in trying it,
i’d love to help her… just e-mail me…
-bowerbird
Hi all,
Jon’s solution, with the addition of the archives section noted above (and after many fits, starts, and more “help!” emails than I’m sure Jon wanted to receive) seems to be working pretty well for now. I’m certainly open to looking at other options, as long as they are something that a non-techie can work with (without driving my brother any crazier than I have already done).
Here’s where we are so far –
http://lettersfromindia1961.wordpress.com/
No photos have been added yet, but I’m going to start working on that soon …. and we do have some absolutely stunning ones. If you’re interested, you can see some of them here, in a Google doc I set up in order to start sharing some of the letters and photos –
http://tinyurl.com/yhb2um8.
You will recognize some famous faces in them.
As I mentioned above, Jon has recommended using the pro version of Flickr to categorize the photos, and then import them into WordPress. I think I’d like to intersperse them with the text, as I did with the Google doc, although I’m certainly open to suggestions on that front as well.
Thanks!
Ruth
I’m noticing a couple of details.
1. slugs and URLs
The most recent pattern is:
Title: July 20, 1961 – Kids’ adjustment (Anita)
URL: http://lettersfromindia1961.wordpress.com/1961/07/20/july-20-1961-kids-adjustment-anita/
The slug, in WP lingo, is this piece of the URL:
july-20-1961-kids-adjustment-anita
These are made automatically from the titles. But they can be adjusted afterward. I would recommend subtracting the redundant dates from them so the URLs come out like this:
http://lettersfromindia1961.wordpress.com/1961/07/20/kids-adjustment-anita/
2. Order of items on tag pages
Here’s the tag page for Galbraith:
http://lettersfromindia1961.wordpress.com/tag/galbraith/
The items come out in WP default order, namely reverse chronological. But your style for this project is to run forward in time.
There is a way to make tag pages do that:
http://lettersfromindia1961.wordpress.com/tag/galbraith/?order=asc
However, there doesn’t seem to be a way to make the WordPress.com tag cloud widget produce links having that form. It would be interesting to know if there’s another way to do it.
i set up my system with letters from your sites,
if you would be interested in taking a look at it.
-bowerbird
Where?
> Where?
my system isn’t yet ready for release to the public.
so please e-mail me to get that location info, jon.
anyone else who wants to mount a similar project
can e-mail me to take a stab with my system too:
> bowerbird@aol.com
-bowerbird
I finally got round to writing up what I had to do to get the Diary of Samuel Pepys website working properly in Movable Type: http://www.gyford.com/phil/writing/2010/07/09/mt-old-dates.php Probably no direct use to you Jon, but possibly for others who might come across this.
Just ran across 1000 Memories, a startup which seems to be interested in at least one dimension of your project: the online memorial for a family member:
http://1000memories.com/
They were in the biggest Y-combinator demo day to date:
http://techcrunch.com/2010/08/24/y-combinator-demo-day-2/
Didn’t look exactly like what you have in mind, but they might be especially motivated to support a project like this.
An interesting dialogue is value comment. I think that you need to write extra on this matter, it won’t be a taboo topic but typically individuals are not enough to speak on such topics. To the next. Cheers