SiMAMT: A Framework for Strategy-Based Multi-Agent Multi-Team Systems

1. Introduction

When multi-agent scenarios move beyond singular, short-term goals and into the realm of multi-layered strategies the complexity quickly scales out of the practical. Much of the research in multi-agent systems revolves around single-goal systems where multiple agents each work independently to achieve the same goal. This does not accurately model the real-world scenarios found in larger systems where each independent agent has their own initiatives but still works together to achieve team goals. These teams are also part of larger teams (e.g., units make up regiments, regiments make up battalions, etc.) that also have large-scale goals. As the hierarchy builds, the strategy becomes larger and extends further. The authors are proposing a framework to address this particular issue. The SiMAMT framework is designed to allow a hierarchical strategy structure that works at each level to enforce policies that work at that particular level. Each sub-level of the hierarchy then works at its’ particular level order while considering the orders filtered down from the higher level. In this manner, the entire structure incorporates a multi-level strategy without having to use a large, monolithic policy (these large policies arise from applying small-scale solutions to much larger problems). When policies are allowed to grow in scale with the number of agents and the complexity of the system they become computationally too complex to be applied, recognized, and changed in real-time. Strategy-based systems utilize group policies to aggregate the policies of individuals into a larger team policy (which the authors refer to as a strategy). Each of these strategies can be grouped together into a larger strategy at the next level. The SiMAMT framework creates a system to setup, model, control, and analyze multi-level strategies such as these.

SiMAMT seeks to model more complex multi-agent systems where there are multiple teams involved. Strategic interactions at this level are challenging and must be considered carefully. Considering the actions of other agents is foundational to strategy.

It has been shown that strategies offer significant performance enhancement to artificially intelligent agents, that strategies can be recognized in real-time when complexity is limited, and that AI’s utilizing strategy inference will outperform their originally superior opponents (Franklin, 2015). In contrast, classical machine learning requires repetitive trials and numerous iterations to begin to form a hypothesis as to the intended actions of a given agent. There are numerous methodologies employed in an attempt to reduce the number of examples needed to form a meaningful hypothesis. The challenge arises from the difficulty created by the diversity of possible scenarios in which the machine learning algorithm is placed. Given enough time and stability a machine learning algorithm can learn reasonably well in a fixed environment, but this does not replicate the real world very accurately. As a result, strategies offer an opportunity to encapsulate much of this policy-space in a compact representation. They have to be learned as well, but they are transmutable to another instance of a similar problem. Additionally, they can be pre-built and then modified to suit the exact situation.