PDI is an intelligent agent architecture and unsupervised learning algorithm for constructing models of an environment in real-time to be used for planning and decision-making by an intelligent agent. A PDI agent is embodied, meaning it exists within and as part of a larger environment, and adaptive, meaning it changes behavior in response to interactions with the world. actions produce changes that are observed by the agent, providing a feedback loop that is utilized to optimize performance. PDI uses specific data structures to convey information at different levels of the architecture. each type of data structure is a vector of fixed size and with roles assigned to each element.
The first type of structure is an observation, which is used to carry information from the environment to the cognitive system of the agent. observations provide data regarding current state, facing state (i.e. that toward which the agent is directing attention) and peripheral states (i.e. those toward which it is possible to direct attention). the second type of structure is a command, which carries signals to motors and prompts them to function in the environment. commands also handle attention, guiding both mental and physical activity. the third type of data is change, which is represented as a set of binary features describing a comparison between some previous observation and the current observation. And finally there’s the transition, which is a pair of states linked by an action, as well as a change vector comparing the first to the second. Transitions directly associate changes that occur in observation to actions performed just before, enabling one to learn a mapping from action to change such that the result of performing some action in some state in predictable.
PDI comes installed with a trained classifier that maps commands to changes. given an action vector, the algorithm will predict its effect on an observation, based on the change that is likely to occur as a result, as well as the state currently observed. Changes are then used to generate predictions, or observations that are imagined or produced internally rather than perceived. predictions are recursively produced, representing events that take place further and further into the future as the agent performs a new prediction by taking the prediction made at the previous step as the new “current state”.
Agents are restricted as is any sufficient intelligence with the cognitive ability to predict and plan for the future, due to the chaos of complex systems. This chaos manifests as inherent unpredictability with respect to long-term forecasting. for this reason agents treat predictions more distant in time as less important, and those closer as more reliable and relevant. in other words, agents favor information immediately relevant, effectively directing their cognitive resources toward more urgent matters.
The predictions made by an agent as described are formed in a bottom-up process that requires no contextual information to yield accurate results. predictions made this way are quite restricted in application, as they are made using simple rule-based mechanics. Such simplicity cannot simulate a world with even moderate complexity, necessitating a second and more advanced knowledge system within the agent to handle complexity with a more flexible and adaptive, yet slower system, that is based on and extends the knowledge generated from the bottom-up to form a vast interconnected graph that re-contextualizes the data learned previously by forming connections between points while taking into account a wider scope of information. The result is a system that uses local activity to activate related states with respect to observations, thus bringing into consciousness a set of possible futures that are reachable from the current state.
Such a system requires the low-level functionality of an agent to produce the data needed to build a solid foundation. through this model, predictions are made from the top-down, starting at the structured knowledge represented by a graph stored in long-term memory, and influencing the observations and actions taking place in the environment. low-level decisions are influenced by signals produced via high-level decisions, which take into account information from the surrounding context, including states that are unobserved but are believed to be reachable, details from experience that are important to current goals, etc. basically all information that is not readily accessible via low-level observation but is still important or useful in some way is preserved in a world model and retrieved upon an experience where the knowledge lends itself to accomplishing an objective.
An agent begins constructing a model using transitions observed over time. the links between states are formed when a pair of them are observed in sequence, and an observable change exists between them. when a state transition is consolidated, two nodes representing the initial and final state of the transition are connected by a synapse, labeled by the action used to get from one to the other. because actions map to changes a-priori, links between nodes labeled with actions are easily translated into changes, which can then be used to make bottom-up predictions if need be.
The essential process of the agent’s memory is it’s two-path prediction system, which enables both bottom-up and top-down predictions to be made simultaneously, each half of the process feeding back and influencing the other. bottom-up generation is the process in which a model is formed, through a rule-based mechanism, and eventually leading to high-level functions that are adaptive and error resistant, and which manage the low-level functions by transmitting signals downward that influence behavior. the interplay between lower and upper cognitive levels permits an intelligent agent to sense, act, and learn online. as its cognitive faculties mature, the agent builds onto its prior knowledge and extends it, preserving all that has been accumulated while making use of it as scaffolding to reach novel ideas.
Building Intelligence 