论文标题
对复杂环境的影响力增强的在线计划
Influence-Augmented Online Planning for Complex Environments
论文作者
论文摘要
我们如何在复杂的环境中进行实时的有效计划，以控制可能涉及许多其他代理的代理？尽管现有的基于样本的计划者在大型POMDP中取得了经验成功，但他们的性能在很大程度上依赖于快速模拟器。但是，实际情况本质上是复杂的，它们的模拟器通常在计算上要求，这严重限制了在线计划者的性能。在这项工作中，我们提出了一种具有影响力的在线计划，这是一种原则性的方法，可以将整个环境的分类模拟器转换为本地模拟器，该模拟器仅示例与计划代理人的观察和奖励最相关的状态变量，并使用机器学习方法捕获了其余环境中的传入影响。我们的主要实验结果表明，与POMCP相比，与对整个环境建模的模拟器进行计划相比，计划不准确但更快的本地模拟器可提高实时计划性能。
How can we plan efficiently in real time to control an agent in a complex environment that may involve many other agents? While existing sample-based planners have enjoyed empirical success in large POMDPs, their performance heavily relies on a fast simulator. However, real-world scenarios are complex in nature and their simulators are often computationally demanding, which severely limits the performance of online planners. In this work, we propose influence-augmented online planning, a principled method to transform a factored simulator of the entire environment into a local simulator that samples only the state variables that are most relevant to the observation and reward of the planning agent and captures the incoming influence from the rest of the environment using machine learning methods. Our main experimental results show that planning on this less accurate but much faster local simulator with POMCP leads to higher real-time planning performance than planning on the simulator that models the entire environment.
