Jon Udell

2010-08-04

As of build 777, the source code is released and I’ve begun a series of articles about the development of the project.

Among other recent activities, I’ve been reviewing how curators are including the default HTML rendering in their sites. Sourcing the HTML into an iframe is a popular approach, as seen for example on berkeleyside. In that situation you’d probably like to suppress the calendar’s header image, since the enclosing page already has one. This was formerly only possible by redirecting to an alternate template, but a lighter solution is now available: use header_image=no in your metadata.

Another strategy for sourcing events into a web page can be seen at InMenlo, where the combined ICS feed from the elmcity aggregator is routed back through Google Calendar, and the results sourced into the page. I’ve noticed a lot of duplicate events in that presentation, and I theorize that it’s happening because the IDs I’ve been generating were unique but not stable on a per-event basis across aggregator runs. So I’ve switched to a strategy that will produce stable unique IDs and will watch what happens. Even if I’m right, it may take a while for the duplicates to scroll off the event horizon.

2010-06-25

Given the hiatus since 607, build 659 clearly includes a lot. Much is internal, as I prepare to release the code as open source. But at least these things are visible:

1. New http://elmcity.cloudapp.net homepage, with these improvements: a) /services/ID is now, instead of 404, a helpful document that names and explains all the subsidiary links b) replacement of the dropdown list with a link to a). (/via Rob Goodspeed)

2. tags now link to category views For example, if an event is in the category music, then the default rendering links music to an url like http://elmcity.cloudapp.net/services/elmcity?view=music (/via Bill Rawlinson)

3. Facebook is now an event source. See: http://blog.jonudell.net/2010/05/07/facebook-is-now-an-elmcity-event-source/.

2010-03-02

Build 607 introduces the new iCalendar validator which Doug Day, who is also the author of the iCalendar component I am using, has graciously provided.

On the stats page now — for example, http://elmcity.cloudapp.net/services/prescottaz/stats — the column formerly labeled valid? is now labeled score. The score is a number from 0 to 100, which nicely represents the idea that the validity of a calendar feed is, practically speaking, not a binary yes/no but rather a point along a continuum.

If you click a value in that column, you’ll invoke the validator for the corresponding feed.

Alternatively, if you want to check the validity of a feed, you can do it directly at http://icalvalid.cloudapp.net.

2010-02-26

Build 605 includes few if any curator-visible changes, but much internal improvement. Especially in the areas of caching and service self-monitoring.

2009-11-09

Build 556 adds a major new source of events for all geographic hubs: Eventbrite. It works the same way as the Eventful and Upcoming sources, and happens automatically, curatators don’t have to do anything extra or different.

2009-10-22

Build 540 adds two new administrative features for curators: log viewing, and metadata viewing.

1: Log viewing

The URL pattern for the log viewer is:

http://elmcity.cloudapp.net/logs/{id}/{minutes}

So, for example, here’s the last 60 minutes of activity for the Keene hub:

http://elmcity.cloudapp.net/logs/elmcity/60

If minutes is more than 500, it’ll become 500.

2: Metadata viewing

The URL pattern for the metadata viewer is:

http://elmcity.cloudapp.net/metadata/{id}

You wouldn’t normally need this. But if you want to make sure that the metadata you’ve specified in Delicious has been accurately and completely transmitted to the elmcity service, you can use this URL to check. It retrieves the mirror of your Delicious metadata that’s stored in an Azure table, plus some computed or looked-up values that aren’t in Delicious:

• events
• population
• feed_count

If you have made a change to your Delicious metadata, you might want to send a Twitter start message to the service in order to start a new cycle. You can watch its progress in your log viewer. Look for entries like these:

 8:53:45 PM info 5872,RD00155D301CF7: Scheduler.StartTaskForId: eLearningEvents
 8:53:45 PM info 5872,RD00155D301CF7: processing hub: eLearningEvents
 8:54:03 PM info 5872,RD00155D301CF7: DoIcal: eLearningEvents

Once you see DoIcal, your metadata should be synched to Azure. You can check your metadata URL to verify.

2009-10-21

Build 538 adds the ability for curators to short-circuit the normal cycle and start a new one. You do this by sending a Twitter direct message to @elmcity_azure. If the message says:

start

then your cycle will restart within 5 minutes.

2009-10-06

Major changes in build 516:

Improved calendar widget in HTML views
The new version, based on the JQuery datepicker, stays in a fixed position as you scroll, and ajusts itself to the closest displayed date.

Hubs using the default template will see this new behavior. Hubs that have altered the default template may want to resynch with it.

RSS views
The following flavors work:

1. http://elmcity.cloudapp.net/services/elmcity/rss

2. http://elmcity.cloudapp.net/services/elmcity/rss?view=government

3. http://elmcity.cloudapp.net/services/elmcity/rss?count=30

4. http://elmcity.cloudapp.net/services/elmcity/rss?view=music&count=10

This flavor should be especially useful for systems, like WordPress.com, that will not display dynamic content but will display RSS feeds. You can see example 4 in action on my blog (scroll down to “Upcoming music in Keene”).

2009-09-16

Build 452 adds new standard views for tag summaries (as HTML and as JSON). So for example:

http://elmcity.cloudapp.net/services/elmcity/tags_html

http://elmcity.cloudapp.net/services/elmcity/tags_json

or

http://elmcity.cloudapp.net/services/elmcity/tags_json?jsonp=tags

2009-08-25

Build 441 includes WHERE and WHAT hubs on the homepage, and streamlines the display of all associated links.

2009-08-06

Build 430 converts internally to UTC times. This mainly benefits topical hubs, since their events cross timezones. The data feeds — XML, JSON, iCalendar — are now all expressed in terms of UTC.

For geographical hubs, the data feeds are also expressed in terms of UTC. However, the HTML view converts to local time as defined by the tz= metadata slot. Since geographical hubs are only using the HTML view, and none (I think) are using the data feeds, there should be no observable change.

2009-07-23

Build 420 adds a new axis of aggregation: topic rather than geographic location.

The mechanism is similar. Basically just what= instead of where= in the delicious metadata.

The first two examples are:

http://delicious.com/eLearningEvents

http://delicious.com/SustainabilityInBusiness

In these cases, there is no aggregation from Eventful and Upcoming, because there is no location. So it’s purely aggregation of a list of iCalendar feeds, which are handled in the usual way.

These topic aggregations aren’t yet reflected on the home page. But you can find them here:

http://elmcity.blob.core.windows.net/sustainabilityinbusiness/SustainabilityInBusiness.html

http://elmcity.blob.core.windows.net/elearningevents/eLearningEvents.html

Similarly for .xml, .json, .ics.

(The directory names is all lowercase. The filenames are whatever you’re using on delicious — most folks use lowercase, but mixed case is possible and both of these are that way.)

2009-06-16

Build 396 tweaks the default HTML rendering. With each day, events are now grouped by these time-of-day labels: Morning, Lunch, Afternoon, Evening, Night, WeeHours.

2009-06-10

Build 390 starts to make category-based views available. The supported types so far: html and xml.

Example URLs:

http://elmcity.cloudapp.net/services/elmcity/xml/sports

http://elmcity.cloudapp.net/services/prescottaz/html/government

http://elmcity.cloudapp.net/services/whyhuntington/html/nightclub%20scene

Any or all of the categorization methods discussed previously can contribute to these views.

Not yet supported:

http://elmcity.cloudapp.net/services/whyhuntington/html/music,fine-arts

(And not yet decided: Does that mean music AND fine-arts? Or music OR fine-arts? And either way, how to express the alternative.)

Bill R: You have some slash-delimited categories, e.g.:

Fairs/Festivals

Those won’t work.

2009-06-05

Build 388 adds a fourth way to categorize events. In any iCalendar app, you can now use these patterns in the Description field:

url=http://www.harlowspub.com

category=music,bluegrass

This is particularly useful for recurring events. As discussed here, recurring events are a great way to build critical mass because your curation effort keeps paying dividends.

One of the events I found when exploring the search page for Keene is the Monday night bluegrass jam at Harlow’s Pub.

Here’s the description I entered into Windows Live Calendar — which also could have been entered into Google Calendar, or any other iCalendar app:

“The Birch Benders host a Bluegrass picking party at Harlow’s Pub in Peterborough every Monday night – 8 pm until they kick us out (11 or so). url=http://www.harlowspub.com category=music”

Here’s the rendered result:

Mon 08:00 PM Bluegrass night with the Birchbenders (recurring events (live)) (music)

The url=… and category=… patterns can occur anywhere in the description.

I hope this not only opens some doors w/respect to categorization, but also w/respect to finding and adding recurring events, which are a very powerful way to build up critical mass.

2009-06-04

With build 386, we have 3 ways to categorize events:

1. If a source iCalendar feed uses the CATEGORIES property, they’ll be included.
2. If all of the events from a feed can be categorized, you can name that category in the Delicious metadata, using category=CATEGORY. All events from the feed will inherit it  in the same way that they all inherit the default clickthrough link specified with url=URL.
3. If all of the events from an Upcoming or Eventful venue can be categorized, you can also name that category in the Delicious metadata. To do that, bookmark the venue URL and use the patterns venue={UPCOMING|EVENTFUL} and category=CATEGORY.

See today’s blog entry for a more detailed explanation.

2009-05-14

Build 335 addresses a problem that Dave Witzel noticed when he used Google Calendar as a viewer for the Falls Church combined ICS feed.

The problem was lack of context around the displayed event. The (imperfect) solution, discussed in this blog post, copies the URL for the event (or the default URL, from the url= tag in the delicious metadata), into the LOCATION field.

The blog post has the whole story.

2009-05-04

Build 324 adds a service to help curators (or anyone) extract the ICS URL from a web page with an embedded Google Calendar. The writeup is here. The bookmarklet that uses the service is here. To use it directly, for a page like http://www.huntingtoncommunitygardens.com/8.html, do this:

http://elmcity.cloudapp.net/gcal2ics/www.huntingtoncommunitygardens.com/8.html

In other words: http://elmcity.cloudapp.net/gcal2ics/ plus the URL of the calendar page without its leading htttp://

2009-04-16

Build 313 introduces a major new feature for curators. Read all about it here — A power tool for curators — and let me whether you’re able to make use of it.

2009-04-10

The first version of the project FAQ is done. If you know somebody who’s interested in becoming a curator, that page has the information you need.

For a gentler introduction, you can point them to this blog entry which describes the minimal setup required to bootstrap an instance of the calendar aggregator for a location.

2009-04-08

To the extent that most people ever subscribe to ICS feeds at all in their personal calendars, I think the best strategy will be to use the aggregator as a conduit to individual feeds that you might want to subscribe — the hockey schedule, the nightclub schedule.

However, build 298 does now publish the merged set of events for each location as a single ICS feed. I’ll be very interested to know who uses these, and how.

I suspect that experiences will vary according to event density. Adding an ICS overlay for Keene into my Outlook calendar actually does seem to be useful, at the current event density. It may already be overwhelming for Ann Arbor or Baltimore.

In any case, the densities will (hopefully) increase for all locations. So I think in the long run the best use of the aggregator will be to collect and promote discovery of individual feeds.

2009-04-07

Build 295 lays the foundation for per-location iCalendar feeds that merge all sources of events for each location, but does not yet publish those feeds.

The visible change is on the home page, where events, population, and events/person are now reported. Because these are future events, the numbers change even if no feeds are added. I’d like to include a sparkline in the summary table to visualize those trends.

2009-04-03

Today’s version, build 287, focuses on stats reporting. The per-location changes are:

1. Total events
2. Population
3. Ratio of the two

So, for example, this is mashablecity (Providence RI) today:

All events 910, population 48779, events/person 0.02

The per-feed changes are:

1. If a feed fails to validate, False (in the Valid? column) is a link to the full validation report
2. Validation errors are captured and reported
3. Loading errors are captured and reported
4. The PRODID (Product ID) of the software that wrote the feed is reported.

2009-03-30

Build 281, deploying now, is the first fully data-driven version. From now on, I should not have to touch any code to add a new city. The workflow is:

1. A new curator claims a delicious account, as Matt Gillooly just did with delicious.com/mashablecity.
2. I bookmark that URL in my own delicious account, using the tag calendarcuration.
3. On the next cycle, the system queries http://delicious.com/judell/calendarcuration, finds one more curatorial account than before, and rolls it in.

In doing this, I sorted out what the minimal requirement is for account metadata. It is simply — in Matt’s case  — the where slot:

where=providence, ri

Everything else is a slot that you can override from the following defaults:

tz=Eastern

title=event+calendar

contact=nobody_yet

img=http://elmcity.info/media/img/keene-night-360.jpg

css=http://jonudell.net/css/elmcity.css

template=http://jonudell.net/tmpl/events.tmpl

lat={computed from where}

lon={computed from where}

radius=5

2009-03-27

This version makes the “today’s events” widgets on the home page dynamically selectable. With more locations on board, it will be interested to be able to compare any two, side-by-side, for a given day.

2009-03-20

The version deploying now includes these changes:

• CSS. Although the rendered HTML is only meant as a reference implementation — to be replaced with stuff like Bill’s doing here, it didn’t need to be quite so butt ugly.
• Baltimore. I’ve added a fourth collection for localist. It’s currently Eventful+Upcoming only, no iCalendar, pending adjustment of the feed tags. Radius is 5 miles because 15 was overwhelming.
• Metadata. I’m persisting the Delicious metadata into the Azure tablestore. Although Delicious is convenient, not everyone will want to use it. Ed Vielmetti in Ann Arbor would rather use a semantic wiki, which is great. The backend will be a neutral repository for various metadata formats, just as the aggregator is a neutral hub for various calendar formats

With 4 locations running, the update cycle is pushing toward an hour. It won’t be too long before I start running out of hours in the day. But that’s what makes this a nice Azure experiment. The work is naturally divisible and distributable. I’m starting to think about how to hand out tasks to a pool of workers, and coordinate them. Really looking forward to that part, it’ll be interesting!

2009-03-17

While implementing Bill’s fix, I found and fixed several problems that were causing both Eventful and iCalendar events to be omitted. So all calendars are now more complete.

I also improved the iCalendar stats page. Now there’s a complete accounting of single events, recurring events, instances of recurring events, and future events.

2009-03-16

Bill Rawlinson noticed that the per-feed links weren’t working. That’s because although I defined the metadata mechanism, and he implemented it at http://delicious.com/whyhuntington, I had neglected to actually wire it up! Thanks for noticing, Bill, I’m testing the fix now.

The idea is as follows. Some iCalendar feeds include per-event URLs but many do not. In those cases, we can specify a catch-all link for the feed. Since the feed will already be bookmarked in Delicious, you can do that by adding a name=value tag like so:

url=http://www.wvpumpkinpark.com/pp-calendar.asp

> I’m testing the fix now.

Hmm. The test triggered the Delicious rate limit. Which is OK, I’ve been needing to implement caching of external services anyway. So now I will.

2009-03-15

The aggregator now includes Upcoming events in and around your location. This feature is controlled by three tags on the metadata URL in your Delicious account. From the Keene example:

radius=15

lat=42.9336

lon=-72.2786strong>2009