Article
Homeostatic reinforcement learning for integrating reward collection and physiological stability.
Efficient regulation of internal homeostasis and defending it against perturbations requires adaptive behavioral strategies. However, the computational principles mediating the interaction between homeostatic and associative learning processes remain undefined. Here we use a definition of primary rewards, as outcomes fulfilling physiological needs, to build a normative theory showing how learning motivated behaviors may be modulated by internal states. Within this framework, we mathematically prove that seeking rewards is equivalent to the fundamental objective of physiological stability, defining the notion of physiological rationality of behavior. We further suggest a formal basis for temporal discounting of rewards by showing that discounting motivates animals to follow the shortest path in the space of physiological variables toward the desired setpoint. We also explain how animals learn to act predictively to preclude prospective homeostatic challenges, and several other behavioral patterns. Finally, we suggest a computational role for interaction between hypothalamus and the brain reward system.
A series of dates of unfolding of the first leaves and duration of the season of vegetation in the silverbirch (Betula pendula Roth. (B. verrucosa Ehrh.)), as well as the duration of flowering of the bird cherry (PaFdus avium), mountain ash (Sorbus aucuparia), andsmall-leaved lime (Tilia cordata Mill.) for the period1970–2010 in the central part of European Russia were studied in order to assess the trends. Differences in phenological responses to homogeneous climate changes in the trees of the same species from the northernand southern parts of the range were revealed. If spring events occur 3–7 days earlierin the northern part, nosuch effect is observed in the south. This fact can be interpreted as a manifestation of the different mechanisms of homeostasis in different populations determined by their biological characteristics (in particular, by the need to pass successfully the periods of organic rest and vegetation).
Celebrity endorsement is omnipresent. However, despite its prevalence, it is unclear why celebrities are more persuasive than (equally attractive) non-famous endorsers. The present study investigates which processes underlie the effect of fame on product memory and purchase intention by the use of functional magnetic resonance imaging methods. We find an increase in activity in the medial orbitofrontal cortex (mOFC) underlying the processing of celebrity–product pairings. This finding suggests that the effectiveness of celebrities stems from a transfer of positive affect from celebrity to product. Additional neuroimaging results indicate that this positive affect is elicited by the spontaneous retrieval of explicit memories associated with the celebrity endorser. Also, we demonstrate that neither the activation of implicit memories of earlier exposures nor an increase in attentional processing is essential for a celebrity advertisement to be effective. By explaining the neural mechanism of fame, our results illustrate how neuroscience may contribute to a better understanding of consumer behavior.
Our annual conference, Neuroeconomics: Decision Making and the Brain, aims to promote interdisciplinary collaborations and discussions on topics lying at the intersection of the brain and decision sciences in the hopes of advancing both theory and research in decision making. To this end, we welcome involvement by all researchers interested in these and related topics, including reward, learning, emotion, and social behavior to name but a few. Our meeting embraces a wide breadth of research; please feel free to download abstracts and other material from our previous conferences below.
The role of cortisol (Crt), dehydroepiandrosterone (DHEA) and DHEAsulfate (DHEAS) in stress responses were shown. The fluctuations in concentration of Crt, DHEA and DHEAS depending on age, sex and time of the day in norm and under acute and chronic stress were quoted. The main techniques of assessment of serum, urine and saliva Crt concentrations were discussed. A special attention had been paid to the use of Crt concentration in anthropological and psychological research. Bibliography comprises 181 works.
We present two examples of how human-like behavior can be implemented in a model of computer player to improve its characteristics and decision-making patterns in video game. At first, we describe a reinforcement learning model, which helps to choose the best weapon depending on reward values obtained from shooting combat situations. Secondly, we consider an obstacle avoiding path planning adapted to the tactical visibility measure. We describe an implementation of a smoothing path model, which allows the use of penalties (negative rewards) for walking through ``bad'' tactical positions. We also study algorithms of path finding such as improved I-ARA* search algorithm for dynamic graph by copying human discrete decision-making model of reconsidering goals similar to Page-Rank algorithm. All the approaches demonstrate how human behavior can be modeled in applications with significant perception of intellectual agent actions.
In this article a combination of two modern aspects of games development is considered: (i) the impact of high quality graphics and virtual reality (VR) user adaptation to believe in realness of in-game events by user’s own eyes; (ii) modeling an enemy’s behavior under automatic computer control, called BOT, which reacts similarly to human players. We consider a First-Person Shooter (FPS) game genre, which simulates an experience of combat actions. We describe some tricks to overcome simulator sicknesses in a shooter with respect to Oculus Rift and HTC Vive headsets. We created a BOT model that strongly reduces the conflict and uncertainty in matching human expectations. BOT passes VR game Alan Turing test with 80% threshold of believable human-like behavior.
Adaptive and Learning Agents Workshop at International Joint Conference on Autonomous Agents and Multiagent Systems
Our decisions are affected not only by objective information about the available options but also by other people. Recent brain imaging studies have adopted the cognitive neuroscience approach for studying the neural mechanisms of social influence. A number of studies have shown that social influence is associated with neural activity in the medial prefrontal cortex and ventral striatum, which are two brain areas involved in the fundamental and not exclusively social mechanisms of performance monitoring. Therefore, the neural mechanisms of social influence could be deeply integrated into our general neuronal performance-monitoring mechanisms.
A model for organizing cargo transportation between two node stations connected by a railway line which contains a certain number of intermediate stations is considered. The movement of cargo is in one direction. Such a situation may occur, for example, if one of the node stations is located in a region which produce raw material for manufacturing industry located in another region, and there is another node station. The organization of freight traffic is performed by means of a number of technologies. These technologies determine the rules for taking on cargo at the initial node station, the rules of interaction between neighboring stations, as well as the rule of distribution of cargo to the final node stations. The process of cargo transportation is followed by the set rule of control. For such a model, one must determine possible modes of cargo transportation and describe their properties. This model is described by a finite-dimensional system of differential equations with nonlocal linear restrictions. The class of the solution satisfying nonlocal linear restrictions is extremely narrow. It results in the need for the “correct” extension of solutions of a system of differential equations to a class of quasi-solutions having the distinctive feature of gaps in a countable number of points. It was possible numerically using the Runge–Kutta method of the fourth order to build these quasi-solutions and determine their rate of growth. Let us note that in the technical plan the main complexity consisted in obtaining quasi-solutions satisfying the nonlocal linear restrictions. Furthermore, we investigated the dependence of quasi-solutions and, in particular, sizes of gaps (jumps) of solutions on a number of parameters of the model characterizing a rule of control, technologies for transportation of cargo and intensity of giving of cargo on a node station.
Event logs collected by modern information and technical systems usually contain enough data for automated process models discovery. A variety of algorithms was developed for process models discovery, conformance checking, log to model alignment, comparison of process models, etc., nevertheless a quick analysis of ad-hoc selected parts of a journal still have not get a full-fledged implementation. This paper describes an ROLAP-based method of multidimensional event logs storage for process mining. The result of the analysis of the journal is visualized as directed graph representing the union of all possible event sequences, ranked by their occurrence probability. Our implementation allows the analyst to discover process models for sublogs defined by ad-hoc selection of criteria and value of occurrence probability
The general aim of this thesis is to explore the gendered and classed nature of social work and social welfare in Russia to show how social policy can be a part of and reinforce marginalisation. The overall research question is in what ways class and gender are constructed in Russian social work practice and welfare rhetoric through Soviet legacies and contemporary challenges? In addition, which actors contribute to the constitution of social work values and how this value system affects the agency of the clients? This study focuses on contradictory ideologies that are shaped in discursive formations of social policy, social work training and practice. It is a qualitative study, containing fi ve papers looking at this issue from three different perspectives: policy and institutions, culture and discourse, actors and identity. The data collection was arranged as a purposive–iterative process. The empirical material consists of qualitative interviews with social work practitioners, administrators and clients, participant observations in social services and analysis of documents of various kinds.
Existing approaches suggest that IT strategy should be a reflection of business strategy. However, actually organisations do not often follow business strategy even if it is formally declared. In these conditions, IT strategy can be viewed not as a plan, but as an organisational shared view on the role of information systems. This approach generally reflects only a top-down perspective of IT strategy. So, it can be supplemented by a strategic behaviour pattern (i.e., more or less standard response to a changes that is formed as result of previous experience) to implement bottom-up approach. Two components that can help to establish effective reaction regarding new initiatives in IT are proposed here: model of IT-related decision making, and efficiency measurement metric to estimate maturity of business processes and appropriate IT. Usage of proposed tools is demonstrated in practical cases.