Tensorizing Neural Networks
Deep neural networks currently demonstrate state-of-the-art performance in several domains.At the same time, models of this class are very demanding in terms of computational resources. In particular, a large amount of memory is required by commonly used fully-connected layers, making it hard to use the models on low-end devices and stopping the further increase of the model size. In this paper we convert the dense weight matrices of the fully-connected layers to the Tensor Train format such that the number of parameters is reduced by a huge factor and at the same time the expressive power of the layer is preserved.In particular, for the Very Deep VGG networks we report the compression factor of the dense weight matrix of a fully-connected layer up to 200000 times leading to the compression factor of the whole network up to 7 times.
The paper analyzes storage peculiarities of satellite Earth remote sensing data time series. We propose methods for their compression based on the discovered peculiarities exploiting different schemes of Huffman coding. One of the proposed methods reaches 6% increase in the compression ratio (93%) in contrast to the deflate method used in Java SE6 (87%), for a time series of aerosol optical thickness derived from MODIS radiometer of TERRA satellite. Further improvement can be achieved by using the entropy coding of floating point numbers.
It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time. We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval.
The technology of storage of poorly formalized documents that are created using lexicological synthesis is considered. The technology provides formation of the kept index sequences containing indexes of document forms and their substantial components. Additionally, thanks to simultaneous preparation of documents and creation of kept index sequences, the economy of time is provided. The experiments have shown the efficiency of the approach for the documents created for management of different kinds of activity.
The book presents the most important aspects of safe digital image workflows, starting from the basic practical implications and gradually uncovering the underlying concepts and algorithms. With an easy-to-follow, down-to-earth presentation style, the text helps you to optimize your diagnostic imaging projects and connect the dots of medical informatics.
The increasing of the efficiency of technological modes of steel products manufacturing requires simulation of metal forming during hot deformation. To obtain correct results, one should set the correct initial and boundary conditions, including the mechanical properties of materials, which represent the dependence of the stress-strain and strain rate at maintained temperature. In the experiments one must reveal the mechanical properties and constants of the steels according to strain rate, predetermined temperature and chemical composition. So, the type of test is usually dependents on the technology process, which simulation will be using the obtained information. One can identify four main types of tests used in the hot deformation: compression, tension, torsion and rupture tests. The simplest tests are considered as uniaxial compression or tension tests. The results of these tests are the curves of <<flow stress -- strain>>. The present study describes an approximation method of test results for uniaxial compression of cylindrical samples made from AISI304 steel. During this work a mathematical model of the <<stress -- strain>> relation has been described. An algorithm that determines the necessary numerical coefficients for this model was developed. As a result, the equation of the material state, which is characterized by the stress relation on the strain, strain rate (0.15, 0.5, 1.5, 5 and 15 inverse seconds) and temperature (800, 950, 1080 and 1200 degree Celsius) was found. Also the approximation comparison with the experimental results were obtained.
In this paper we compare the Russian National Corpus to a larger Russian web corpus composed in 2014; the assumption behind our work is that the National corpus, being limited by the texts it contains and their proportions, presents lexical contexts (and thus meanings) which are different from those found ‘in the wild’ or in a language in use.
To do such a comparison, we used both corpora as training sets to learn vector word representations and found the nearest neighbors or associates for all top-frequency nominal lexical units. Then the difference between these two neighbor sets for each word was calculated using the Jaccard similarity coefficient. The resulting value is the measure of how much the meaning of a given word is different in the language of web pages from the Russian language in the National corpus. About 15% of words were found to acquire completely new neighbors in the web corpus.
In this paper, the methodology of research is described and implications for Russian National Corpus are proposed. All experimental data are available online.