To keep the data mart current, an automated Unix shell script captures all the relevant data and transmits it to Interelate via SCP, a secure communications utility that uses Secure Shell (SSH) as its underlying protocol. SCP and SSH work like Unix "cp" and "sh" commands, but use public key cryptography to ensure that data is transferred securely over unsecured communication channels. All the changes to the database in the previous 24 hours are captured and transmitted to Interelate early every morning.
"It typically takes about 10 minutes to extract, compress, and transmit the data," Doyel says. "The key to minimizing this time was designing the system for daily, incremental updates instead of complete copies of the database. If we were to transmit a complete copy of the database it would take a full day at top bandwidth."
Lisa Napell Dicksteen is president of LMN Editorial, which handles writing and editing as well as marketing and public relations consulting.