On Sunday, in a New York Times story about Popfly, John Markoff wrote:
Because the company chose to design Popfly using a Microsoft Web graphics and animation technology called Silverlight, it will be treated with suspicion by an Internet universe that is increasingly committed to open standards.
Disclaimer: I work for Microsoft, and John Montgomery, who leads the Popfly project, has been a friend since our days together at BYTE. That said, I think this overstates Popfly’s relationship to Silverlight. Although the Popfly designer runs in Silverlight, mashups created in Popfly don’t require it. Most are just made of plain old HTML and JavaScript.
Elsewhere in the article, this quote from Tim O’Reilly appears:
Popfly shows me that Microsoft still thinks this is all about software, rather than about accumulating data via network effects, which to me is the core of Web 2.0. They are using Popfly to push Silverlight, rather than really trying to get into the mashup game.
Silverlight, as I’ve said, isn’t Popfly’s focus. I do agree that Popfly doesn’t operate in the cloud in the same way as, say, Yahoo! Pipes. While the article doesn’t mention Pipes, I often hear Popfly and Pipes mentioned in the same context. Both are mashup creators, but they are architecturally very different — in complementary ways. Because Pipes is a great example of data-oriented network effects, and because I’ve sometimes confused myself about the differences between Popfly and Pipes, it’s helpful to spell them out.
Mashup engine
Popfly’s mashup engine is a hybrid. There’s a service running in the cloud, but your browser can do a lot of work too.
Pipes’ mashup engine lives entirely in the cloud.
Sharing and discovery
Both systems provide a cloud-based environment for sharing and discovering mashups, and the components of mashups.
Inputs
Both systems can mash up data flowing from a variety of services on the web, including those that produce RSS feeds and other kinds of XML outputs.
Outputs
Popfly ends at the glass. The output is an HTML/JavaScript page or widget that renders in your browser. Although the components used to produce that output live in the cloud, the final result ends in your browser and isn’t available for downstream processing in the cloud.
Pipes can keep going. The output is a data feed that may or may not drive a browser-based display. But even when it does, the output is still available for downstream processing in the cloud — for example, as RSS.
Programming
In Popfly, you do basic stuff with a special-purpose visual programming language that runs in the cloud. You do advanced stuff with JavaScript running in the browser.
In Pipes, you only use a special-purpose visual programming language that runs in the cloud.
It gets confusing, even to me, because you can sometimes use both systems to achieve the same result. Consider this FluxnetTowers mashup in Popfly, which maps the locations of a worldwide network of C02 flux sensors. I just now made a simplified version in Pipes. I’m sure it’s possible to reproduce the annotations shown in the Popfly version. But from my perspective it’d be harder, because Pipes lacks the general-purpose programming available in Popfly thanks to JavaScript.
Suppose you wanted to include that same tower data in a widget on your WordPress blog, though. Here, Pipes would be the choice. WordPress lives in the cloud, and so does Pipes, so you can make Pipes produce a feed that WordPress consumes. But you couldn’t use Popfly in this scenario because a cloud-based service like WordPress can’t access the output of Popfly’s browser-based mashup engine.
Pipes likes to aggregate, transform, and filter data feeds within the cloud, and can produce a few kinds of renderings in your browser. Popfly likes to aggregate, transform, and filter data feeds from the cloud, and can produce arbitrary renderings in your browser. They’re complementary because Popfly can consume and render data feeds coming from Pipes.
Yes, it’s also worth pointing out that PopFly has the ability to consume data that is not accessible by the cloud – i.e. data behind the firewall.
For some, this is a key factor to mashing up internal restricted data, with external publish domain information.
There’s also no real technical reason why you couldn’t proxy a Popfly mashup from a server. The Popfly architecture is therefore more versatile.
I updated this post to clarify two points:
a) Popfly’s mashup engine is a hybrid of a cloud-based service and the browser. The original version omitted the former and thus overstated the role of the latter.
b) Popfly’s programming model is likewise a hybrid. Whether you see it as a mainly a visual programming environment or mainly a JavaScript environment depends on your background and experience. From my perspective, I see it mainly as a JavaScript environment. That’s where you create new blocks.
“ability to consume data that is not accessible by the cloud – i.e. data behind the firewall”
Interesting point, hadn’t considered that.
“There’s also no real technical reason why you couldn’t proxy a Popfly mashup from a server”
I guess, but to do it in a complete way would seem to require server-side JavaScript, at which point it’s kind of a new ballgame.
“The Popfly architecture is therefore more versatile.”
I think it’s tricky to make such characterizations. Either system can appear more or less versatile depending on what you need to do, and what kinds of skills you bring to the table.
I personally have not tried Popfly, because it requires Silverlight.
I make great use of Yahoo Pipes, primarily as a filter for RSS feeds. Some feeds insert ads, which I filter out (especially those that have graphics in the ad).
I also use the Pipes to filter the RSS feeds from bargain sites, such as Slickdeals.net, because these sites produce way more bargains than I want to see, so I can create a new, filtered feed for each item I might be looking to purchase. For example, I have created feeds for a specific MP3 player from any store and a feed for sale dog food from a specific store.
My favorite Pipes feed is a filter for the Netflix New DVD Releases RSS feed, where I see only new movies and TV Shows I want (after making a simple insertion of the correct info into Pipes), instead of the massive number of new releases.
“I personally have not tried Popfly, because it requires Silverlight”
What’s your take on a) Silverlight, and b) Popfly’s use of Silverlight?
“I make great use of Yahoo Pipes, primarily as a filter for RSS feeds”
This is absolutely the sweet spot for Pipes. Yesterday, via earideas.com, I found WNYC’s Radio Lab.
http://earideas.com/earideas/explore/feed/176/WNYC's+Radio+Lab
The RSS feed listed at earideas turns out to be a Y Pipe:
http://pipes.yahoo.com/pipes/pipe.info?_id=6DLRYYLO3BGfTM83JphxuA
Investigating, I found it had been used to solve a really common problem. The site wound up with two different podcast feeds and needed to splice them together. Could be done in FeedBurner, but this was a quicker/easier solution. Interesting to see the WNYC blog relying on Pipes in this way to create, and also publish, its combined feed.
Another great example is here:
http://blog.jonudell.net/2007/10/10/combining-tagspaces/
I’d asked about combining tagspaces from different bookmarking services, and engtech solved it neatly with a Y Pipe.
When data flows are in the form of RSS feeds, Pipes excels at combining/filtering/republishing.
With Popfly, OTOH, you can more effectively parse and filter data sources that don’t happen to be in RSS format. Its JavaScript environment, for example, provides an XPath search that abstracts the differing XPath APIs of IE and FF. You can’t republish that data as a feed because that’s not what Popfly does. You can, however, merge that data into AJAX-enhanced HTML rendered in the browser, which is Popfly’s raison d’etre but not that of Pipes.
Like I said: similar-but-different, and complementary.
“I guess, but to do [cloud based popfly mashups] in a complete way would seem to require server-side JavaScript, at which point it’s kind of a new ballgame.”
But isn’t that exactly what “democratizing the web” is all about? JScript -> IL -> ServerSide. At least for non-HTMLDOM based java script I can’t see any real technical hurdles. I prefer this model to the cloud only model at least.
Jon, I’d be interested in a similar comparison between Google Mashup Editor and Popfly, as GME seems to be a more apples-to-apples comparsion with Popfly (at least at first glance), and doesn’t require a framework install. Popfly’s designer is I’m sure jazzier, but I don’t have to install Flash, Silverlight, or anything else to use GME.
I’d like to see that same comparison between GME and Popfly.