Programmatic agents and causal state abstractions

Rasmus Larsen

Research output: Book/ReportPh.D. thesis

60 Downloads (Pure)

Abstract

When people think of artificial intelligence, they might imagine the kind of anthropomorphic robots found in science-fiction literature and films. These robots interact with the world in a way that comes naturally to humans, and like humans they have the ability to navigate through new situations by using knowledge and skills that they have previously acquired.

The methods that we use today for training artificial agents that interact with their environment lack several key features, which are needed to achieve the mentioned generalization ability. These features are composition, communication, and abstraction; in short, composition is to form a structure from smaller parts, communication is the ability to be understood by humans and other agents, and abstraction is the ability to look past details and see the bigger picture. When applied to the behavior of agents, these features allow for flexible behavior that combines previously learned skills to new situations, while also being able to tell others about the whys and hows of the behavior. A lot of this is reflected in the natural languages that humans use to communicate, and both these and languages used to write computer programs are structured with composition and communication in mind. The first contribution is a method for learning a representation of agent behavior encoded in a computer language, by imitating an existing policy which does not have a language representation. Since this language representation is composed of smaller parts, and
can be read by people or machines, it is a step towards the features of composition and communication.

The work towards the third feature, abstraction, is based on the concept of causality. The basis of causality is straightforward: if event B happens because event A happened, then we say that A caused B. This concept is useful because it can inform us about whether something is important or not, either because it has an influence on something we care about, or because it does not. The second contribution is a method for learning a function that an artificial agent can use to group a bunch of distinct yet similar observations into a single state. For example, when opening the front door of a house, it might be useful for an agent to refer to having a key to the front door of the house simply as “having a key”, and not “having a key while cooking dinner in the kitchen”, or “having a key while cleaning the north-west corner of the living room” – essentially, the causality principle is used to infer what is important to represent, due to it having a causal impact on the goal of opening the door.
Original languageEnglish
PublisherTechnical University of Denmark
Number of pages108
Publication statusPublished - 2021

Fingerprint

Dive into the research topics of 'Programmatic agents and causal state abstractions'. Together they form a unique fingerprint.

Cite this