There is a growing trend towards increased channel density of data acquisition and data collection systems. As a result, the amount of recorded data requires new techniques and technologies to manage effectively.
Several roadblocks can get in the way of optimal data processing. The data may as well never be collected at all if it is stored without descriptive properties, in inconsistent formats, or scattered about on arrays of computers. These factors combine to create a graveyard of information that makes it extremely difficult to locate the right data set and derive decisions from it.
Off-the-shelf computing technologies, built on industry standards, are making it easier and less expensive than ever before to manage large data sets. Topics covered include XML data storage schemes, redundant arrays of inexpensive disks (RAID), gigabit Ethernet communication, PCI Express data transfer, and data mining techniques.