This project is part of the Research Cluster CINEMENTAS (“Collaborative Intelligence Based on Mental Models of Assistive Systems”) funded by a major international technology company. We investigate the role of mental models for interactive reinforcement learning, according to which learning is seen as a  dynamic collaborative process in which the trainer and learner together trying to figure out best action policy. We study how learning of a cognitive system can be sped up when the human trainer gives evaluative, informative feedback based on mental models of the system and the learning process, and the learner uses this feedback to build a (potentially simplified) model of the task domain and to shape its policy.