Perspectiva Retrospectiva Repetición de experiencia (HER) is an advanced technique used in aprendizaje por refuerzo, particularly in environments where agents learn from trial and error. The core idea behind HER is to enable an agent to learn from its past experiences even if the outcomes were not as desired. This is particularly useful in recompensas escasas entornos, donde un agente puede tener dificultades para recibir retroalimentación por sus acciones.
En el aprendizaje por refuerzo tradicional, un agente aprende interactuando con el environment and receiving rewards based on its actions. However, in many scenarios, an agent may take actions that do not lead to immediate rewards, making it difficult to learn effectively. HER addresses this challenge by allowing the agent to revisit its past experiences and reinterpret them in a more beneficial way.
The process involves storing a record of the agent’s experiences, including the states it visited, the actions it took, and the resulting outcomes. When the agent fails to achieve its original goal, HER allows it to ‘replay’ these experiences, but with a different goal in mind. This means that the agent can learn from what it did right or wrong in achieving alternate objectives, thereby enriching its learning process.
By leveraging this approach, HER effectively increases the amount of useful data available for training, leading to faster and more efficient learning. It is particularly popular in robotic manipulation tasks and games where the agent must learn complex comportamientos a partir de retroalimentación limitada.