Research on Improved Method of Storage and Query of Large-Scale Remote Sensing Images

2. Related Work

In the past few years, some efforts have been made to achieve efficient storage and processing through high performance cloud computing technology. Zheng, and Fu (2013) used the Hadoop platform to store vector space data. This method effectively solves the problems of single node failure and lack of extensibility. However, the problem of generating a lot of small files in the process of storing data is not considered, which results in a large amount of memory being used to increase the burden on the primary node. This is because HDFS is a bottleneck caused by managing the metadata of multiple slaves (DataNodes) based on a single master server (NameNode). Especially in processing a large number of small files, in order to improve processing efficiency, the NameNode stores the entire metadata of the HDFS in the main memory. Zhong et al. (2011) proposed a distributed key-value storage model to manage massive image data and store small files in large data files, which effectively solved a large number of small file problems. But, it doesn’t describe of each layer of image metadata, thus increasing the data retrieval time. Chi et al. (2013) designed metadata for each layer of image and used MapFile to store the images generated during the data merge process, thus improving the accessibility of the data. Patel, and Mehta (2015) proposed approach combines the correlated files into one single file to reduce the metadata storage on Namenode to improve the access efficiency of small files. The above methods all store remote sensing data directly on HDFS, and HDFS supports update operations on files is poor. Because remote sensing images have the characteristics of multiple phases, it is necessary to update the data information at different times in the same region in real time. Therefore, the above method cannot directly update the data information of the corresponding location.