Retrospectiva Repetição de Experiência (HER) is an advanced technique used in aprendizado por reforço, particularly in environments where agents learn from trial and error. The core idea behind HER is to enable an agent to learn from its past experiences even if the outcomes were not as desired. This is particularly useful in recompensa escassa ambientes, onde um agente pode ter dificuldades em receber feedback por suas ações.
Na aprendizagem por reforço tradicional, um agente aprende interagindo com o environment and receiving rewards based on its actions. However, in many scenarios, an agent may take actions that do not lead to immediate rewards, making it difficult to learn effectively. HER addresses this challenge by allowing the agent to revisit its past experiences and reinterpret them in a more beneficial way.
The process involves storing a record of the agent’s experiences, including the states it visited, the actions it took, and the resulting outcomes. When the agent fails to achieve its original goal, HER allows it to ‘replay’ these experiences, but with a different goal in mind. This means that the agent can learn from what it did right or wrong in achieving alternate objectives, thereby enriching its learning process.
By leveraging this approach, HER effectively increases the amount of useful data available for training, leading to faster and more efficient learning. It is particularly popular in robotic manipulation tasks and games where the agent must learn complex comportamentos a partir de um feedback limitado.