Dealing with large volumes of data

The pursuit of speed isn’t the only place where an upsizing strategy can be upset. Another common problem resides in the user interface. It’s only too easy to create an application that works well with a small volume of test data but then fails to perform when repointed at a remote database with large volumes of information.

One example, as mentioned earlier, is that very useful user interface widget, the list box. It can be tempting to provide list boxes and drop-down lists whenever a selection is required that is supported by a lookup table in the database. The problem with this approach is that in a test system you might have only a few entries that need to be added to the list. In a production environment, there can easily be thousands (even tens of thousands) of entries. Where a form appears quickly on the user interface in the desktop environment, there could be a wait of several minutes when it’s repointed to a production environment, simply because of the time required to populate the list. Even if this wait is acceptable, having to scroll through an enormous list certainly won’t be.

Therefore, a good deal of thought has to be applied to the user interface to ensure that the interface will continue to be useful when the application is finally deployed. It might be necessary to replace some of the user interface widgets with more complex QBE-style (query by example) lookup forms. Or, alternatively, the drop-down lists might have to be eliminated altogether and replaced with a simpler text box with a validation routine.