The Deep Weight Prior
Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution. In this work, we propose a new type of prior distributions for convolutional neural networks, deep weight prior (DWP), that exploit generative models to encourage a specific structure of trained convolutional filters e.g., spatial correlations of weights. We define DWP in the form of an implicit distribution and propose a method for variational inference with such type of implicit priors. In experiments, we show that DWP improves the performance of Bayesian neural networks when training data are limited, and initialization of weights with samples from DWP accelerates training of conventional convolutional neural networks.
We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout. We extend Variational Dropout to the case when dropout rates are unbounded, propose a way to reduce the variance of the gradient estimator and report first experimental results with individual dropout rates per weight. Interestingly, it leads to extremely sparse solutions both in fully-connected and convolutional layers. This effect is similar to automatic relevance determination effect in empirical Bayes but has a number of advantages. We reduce the number of parameters up to 280 times on LeNet architectures and up to 68 times on VGG-like networks with a negligible decrease of accuracy.
The law of accelerating returns can be viewed as a concept that describes acceleration of technological progress. The idea is that tools are used for developing more advanced tools that are applied for creating even more advanced tools etc. A similar idea has been implemented in algorithms for advancing artificial intelligence. In this paper, the results of applying these algorithms in games are discussed. Nevertheless, real life tasks seem more complicated. The game theoretic approach can be applied for transition from theoretical and unrealistic games to more complex and practical tasks. Applications of the game theoretic approach to advance artificial intelligence in solving tasks in the credit industry are proposed.
This volume contains the papers presented at the session "Data Science" within the V International Conference on Information Technology and Nanotechnology (ITNT-2019). The conference was held in Samara, Russia, during May 21-24, 2019 (itnt-conf.org). The conference is a forum for leading researchers from all over the world aimed to discuss the latest advances in the basic and applied research in the field of Information Technology and Nanotechnology. It is also aimed to attract young people to advanced scientific research and share the latest trends in training and research programs for future ITNT specialists . In addition to the session "Data Science", ITNT-2019 also included three other sessions: "Computer Optics and Nanophotonics", "Image Processing and Earth Remote Sensing" and "Mathematical Modeling of Physico-Technical Processes and Systems". The whole forum brought together more than 450 scientists from United Kindom, Japan, Switzerland, Iran, Poland, Bulgaria, Finland, China, Kazakhstan and Russia, as well as representatives of global high-tech corporations, developers of modern electronics – Huawei, Nvidia, Intel, and Azimuth Photonics, and more than 60 cities in the world. 436 talks enabled discussion on a wide range of topics. The topics of the session "Data Science" were grouped into the following key directions: Data Mining (Big data, Systems and platforms, Methods); Machine Learning (Neural networks, Statistical methods, Feature-based classification, Applications); Security, Cryptography (Cryptosystems design and analysis, Mathematical and algorithmic aspects, Efficient implementations of algorithms, Network security); High Performance Computing (Parallel programming models and languages, Highperformance implementations, Complex systems simulation).
The Fifth HCT Information Technology Trends (ITT 2018) is a major international research conference for the presentation of innovative ideas, approaches, technologies, research findings and outcomes, best practices and case studies, national and international projects, institutional standards and policies on Emerging Technologies for Artificial Intelligence. ITT 2018 will provide an outstanding forum for researchers, practitioners, students, policy makers, and users to exchange ideas, techniques and tools, raise awareness and share experiences related to all practical and theoretical aspects of Emerging Technologies for Artificial Intelligence, so as to develop solutions related to communications, computer science and engineering, control systems as well as interdisciplinary research and applications.
L’ouvrage d’Adrian Mackenzie, professeur au Département de sociologie à l’Université de Lancaster, est d’un genre inédit au sein de la littérature émergente, mais encore peu étendue en sciences humaines et sociales, qui explore le fonctionnement du machine learning (ML). Les avancées spectaculaires de cette branche de l’intelligence artificielle (IA) depuis quelques années ont éclipsé les autres approches en la matière et ont soudainement transformé l’IA en un problème social et politique. Plusieurs auteurs ont déjà insisté sur la nécessité de focaliser le regard sur les outils de l’IA, en pointant les limites des travaux qui ne traitent que des effets sociaux des « algorithmes ». Comme le fait remarquer l’anthropologue des sciences et des techniques Nick Seaver, la plupart des travaux sur le sujet s’agitent au sujet des « algorithmes » ou le « big data », en insistant sur leurs effets néfastes, voire catastrophiques, pour la société sans jamais préciser exactement ce qu’ils sont. Le transfert des connaissances et des perspectives entre les spécialistes en IA et en SHS (d’ailleurs dans les deux sens) est pourtant indispensable pour en proposer une critique informée et efficace.
A search for CP violation in the Cabibbo-suppressed D0 → K+K−π+π− decay mode is performed using an amplitude analysis. The measurement uses a sample of pp collisions recorded by the LHCb experiment during 2011 and 2012, corresponding to an integrated luminosity of 3.0 fb−1. The D0 mesons are reconstructed from semileptonic b-hadron decays into D0μ−X final states. The selected sample contains more than 160 000 signal decays, allowing the most precise amplitude modelling of this D0 decay to date. The obtained amplitude model is used to perform the search for CP violation. The result is compatible with CP symmetry, with a sensitivity ranging from 1% to 15% depending on the amplitude considered.