Improvements to Efficient Retrieval of Very Large Temporal Datasets with the TravelLight Method

Abstract

A considerable number of domains deal with large and complex volumes of temporal data. The management of these volumes, from capture, storage, search, transfer, analysis and visualization, still provides interesting challenges. One critical task is the efficient retrieval of data (raw data or intermediate results from analytic tools). Previous work proposed the TravelLight method which reduced the turnaround time and improved interactive retrieval of data from large temporal datasets by exploring the temporal consistency of records in a database. In this work we propose improvements to the method by adopting a new paradigm focused in the management of time intervals instead of solely in data items. A major advantage of this paradigm shift is to enable the separation of the method implementation from any particular temporal data source, as it is autonomous and efficient in the management of retrieved data. Our work demonstrates that the overheads introduced by the new paradigm are smaller than prior overall overheads, further reducing the turnaround time. Reported results concern experiments with a temporally linear navigation across two datasets of one million items. With the obtained results it is possible to conclude that the improvements presented in this work further reduce turnaround time thus enhancing the response of interactive tasks over very large temporal datasets.