Article Preview
Top1. Introduction
With the rapid development of science and technology, the relationship between science and technology is getting closer and closer while their boundary is becoming more and more blurred. Relevant researches on sci-tech data, especially on sci-tech literature, are of great significance for understanding the connection between sci-tech literature, and assessing their achievements and the level of innovation. A large number of studies have shown that correlation analysis of sci-tech literature is useful for understanding sci-tech trend, measuring innovation level, etc. (Zhiyuan, Aryya, George, Michael, and Claire, 2007; Congfeng, Junming, Dongyang, Yumei, and Lifeng, 2018). The environment and the characteristics of big data make necessary and the valuable analysis on the correlations between the contents of sci-tech literature, which is also a new perspective and new application direction of the sci-tech information analysis. Taking sci-tech papers and patents for example, papers and patents are the two most important types of sci-tech Literature. Paper is an important output of the natural science and the social science research while the information contained in patents is more practical, technical and innovative than that in other sci-tech literature. The traditional academic evaluation standards based on citation methods are not in line with the needs of the big data era.
Deep learning, as a branch of machine learning and a hot field of artificial intelligence research, is intended to establish a mechanism in simulation of the neural network of human brain to analyze, learn, and interpret data such as images, sounds and texts. The concept of deep learning was put forward (Hinton and Salakhutdinov, 2006; Silver, Huang, and Maddisonc, 2016). Based on the deep belief network, the unsupervised, greedy and layer-wise training algorithm was put forward, bringing hope to solve the optimization problems related to deep structure. Later on, the deep structure of multi-layer auto-encoder was proposed. The deep machine learning methods can mainly be divided into two types: supervised learning or unsupervised learning. The learning models constructed under the frameworks of supervised and unsupervised learning are different. For examples, the convolutional neural network is a machine learning model under the framework of deep supervised learning, while the deep belief network is a machine learning model under the framework of unsupervised learning. “Deep learning” is relative to “shallow learning” methods such as support vector machine, lifting scheme and maximum entropy. The shallow learning relies on the artificial experience to extract sample characteristics, and after the model learning, it obtains the single-layer features with no hierarchical relations. While the deep learning can get hierarchical feature representation through automatic learning, using multi-layer feature transformation of input data signal to realize the change of original data sample feature representation to the new feature space, thus facilitating data classification or visualization of features. The deep network structure acquired through deep learning has the characteristics of neural networks. Thus, deep networks are actually neural networks at deep levels, namely deep neural networks (DNN). In recent years, continuous development and innovation has taken place in the deep learning field, which fuels new discoveries in its research and applications, and much progress has been achieved in model, algorithm and large-scale applications. At present, the increasing scale of sci-tech literature data poses a great challenge to the information analysis, and calls for application of the deep learning technology in the sci-tech literature data analysis.