ShopWiki technology: Wiki, Mac Attack and Open Source
Kevin Ryan, Dwight Merriman (DoubleClick's former CEO and former CTO, respectively) and Eliot Horowitz founded ShopWiki in
While ShopWiki operates within a crowded comparison shopping space and has the added challenge of developing and maintaining an active user community, the company believes the ShopWiki technology platform will set it apart. The company describes its technology platform and philosophy below, in their own words.
THE WIKI In its purest form, a wiki is an open collaborative content-editing environment. In its modern incarnation, just about every wiki implements authentication/authorization and document history, and many have the bells and whistles that encroach into content management (auto TOC generation, typing and metadata, auto-aggregation, etc.). A wiki can be thought of as a filesystem which supports arbitrary metadata, a permission system and audit trail, with automatic versioning, complete with rollbacks and diffs. This makes it a shoe-in for keeping config data. So is our on-call schedule, our NOC alerting configuration, and tons of the config for our crawlers.
We are using a customized version of the Tiny MCE Rich Text Editor and wrote the backend for the wiki and the wiki syntax translation system ourselves (with the syntax modeled after that of MediaWiki's) but there's no reason that this couldn't be done with any opensource wiki and some customization.
MAC ATTACK We have zillions of XServes powering our enterprise: crawlers, indexers, and UI servers. Our crawlers are dual G5s with a mirrored pair of 10,000rpm drives with the system and mysql on it, and one scratch disk, while our indexers and UI servers are cluster nodes.OPEN SOURCE We are big fans of open source, and of open standards. This stands to reason when you put your faith in massively collaborative efforts to bring value to projects. We don't involve open source in everything we do, however. For example, we don't use apache (or thttpd, or mathopd, etc.) for our webservers, nor did we use one of the myrriad existing open source wiki projects out there. It just didn't make sense for us. All of our crawling and extraction code is very closed.
Here are some of the open source projects we are investing in:- We're comfortably keeping scores of millions of rows in MySQL.
- All of our logging is built on Log4j.
- All our classes are tested with JUnit.
- Our DNS server is powered by the org.xbill.DNS dnsjava classes.
- Primitive Collections for Java is very helpful.
SEE PART I OF THIS STORY: DoubleClick veteran Kevin Ryan wants to pay you $50 for your video