Inspired by human learning mechanisms, a novel meta-heuristic algorithm named human learning optimization (HLO) is presented in this paper in which the individual learning operator, social learning operator, random exploration learning operator and re-learning operator are developed to generate new solutions and search for the optima by mimicking the human learning process. Then HLO is applied to solve the well-known 5.100 and 10.100 multi-dimensional knapsack problems from the OR-library and the performance of HLO is compared with that of other meta-heuristics collected from the recent literature. The experimental results show that the presented HLO achieves the best performance in comparison with other meta-heuristics, which demonstrates that HLO is a promising optimization tool.
One of the well-known problems in single machine scheduling context is the Job Sequencing and Tool Switching Problem (SSP). The SSP is optimally sequencing a ﬁnite set of jobs and loading restricted subset of tools to a magazine with the aim of minimizing the total number of tool switches. It has been proved in the literature that the SSP can be reduced to the Job Sequencing Problem (JSeP). In the JSeP, the number of tool switches from the currently processed job to the next job depends on the sequencing of all predecessors. In this paper, the JSeP is modeled as a Traveling Salesman Problem of Second Order (2-TSP). We call the induced JSeP by 2-TSP as the Job Sequencing Problem of Second Order (2-JSeP) with a different objective function formulation from JSeP and prove that 2-JSeP is NP-hard. Then the Assignment Problem of Second Order (2-AP) and Karp-Steele patching heuristic are incorporated to solve 2-JSeP. The obtained solution, however, does not guarantee the optimal sequence and are used to seed a Dynamic Q-learning-based Genetic Algorithm (DQGA) to improve the solution quality. Q-learning, which is a kind of reinforcement learning method, is used to learn from the experience of selecting the order of mutation and crossover operators in each generation of the genetic algorithm. The computational results on 320 benchmark instances show that the proposed DQGA is comparable to the state-of-the-art methods in the literature. The DQGA even outperforms the existing methods for some instances, as could improve the reported “best-known solutions” in notably less time. Finally, through the statistical analysis, the performance of DQGA is compared with those of non-learning genetic algorithms.
Recurrent neural networks have proved to be an effective method for statistical language modeling. However, in practice their memory and run-time complexity are usually too large to be implemented in real-time offline mobile applications. In this paper we consider several compression techniques for recurrent neural networks including Long–Short Term Memory models. We make particular attention to the high-dimensional output problem caused by the very large vocabulary size. We focus on effective compression methods in the context of their exploitation on devices: pruning, quantization, and matrix decomposition approaches (low-rank factorization and tensor train decomposition, in particular). For each model we investigate the trade-off between its size, suitability for fast inference and perplexity. We propose a general pipeline for applying the most suitable methods to compress recurrent neural networks for language modeling. It has been shown in the experimental study with the Penn Treebank (PTB) dataset that the most efficient results in terms of speed and compression–perplexity balance are obtained by matrix decomposition techniques.