Massive data crunching on a laptop

Demo 2006: Panoratio Database Images CEO Brett Kirkpatrick demoed a unique technology that allows huge databases to run in-memory on standard PCs will full fidelity--no data loss. He showed a 300 megabyte database of every plan and game in the NFL for a century that is transformed into 10 megabytes. The database has 110 different dimensions and 330,000 events. He also showed a 300 gigabyte Oracle database of pharmacy data--consisting of 126 million prescriptions by 10 million pharmacists and 230,000 drugs over 3 years--transformed to 290 megabytes.

The secret sauce is PDI (Portable Database Image), which can reduce database footprints by 10 to 1000 times. The technology involves finding for patterns in the data, taking out repetitious patterns and using mathematics to represent the differences, greatly reducing file sizes. The Panoratio .pdi doesn't involve any pre-calculations, can integrate with various business intelligence tools in addition to its own Explorer tool.   The company has ten production customers in Europe, Kirkpatrick said, and recently signed on Yahoo in the US. The technology came out of the Siemens AG Artificial Intelligence/Machine Learning Laboratory.