Reasons for the high score in maze2d environment:

[L1U-C]

At the planning stage, is the simulation process of maze2d environment consistent with that of the diffuser? If so, the number of planning steps required to reach the end of the maze should be basically determined by the value of planning horizon. But AdaptDiffuser seems to get a significantly higher score than diffuser. How to understand the mechanism of high score? Thank you.

[Liang-ZX]

Thanks for the question. We have rolled out many different trajectories with different lengths using Diffuser during sampling, some are longer while some are shorter. And we only select those shorter path for data augmentation, then AdaptDiffuser can achive higher score with shorter path.

Besides, the planner does not plan only once to reach the goal point. Instead, it requires to plan multiple times. So, the planning steps are not determined by the value of planning horizon.

[L1U-C]

Thank you for the reply. I'd like to follow up with a more specific question.
Based on my reading of the Diffuser codebase, the Maze2D execution works as follows: the diffusion model generates a fixed-length trajectory of horizon H using “inpainting” to condition on start and goal states, then actions are extracted via a PD controller from consecutive state pairs and executed open-loop, and the agent accumulates +1 reward per step after reaching the goal until the maximum episode length.
Under this setup, the total return is primarily determined by how early the agent reaches the goal, which seems largely governed by the fixed horizon H. If both Diffuser and AdaptDiffuser use the same H and both successfully reach the goal, the resulting scores should be relatively close.
I guess the open-loop execution using PD controller-derived actions may introduce tracking errors, causing Diffuser to actually need more than H steps to reach the goal, and AdaptDiffuser's mechanism could alleviate this issue. Since I don't have access to the Maze2D environment code, my understanding may be incorrect. Could you please explain this issue? Thank you!
Looking forward to your explanation and help！

[Liang-ZX]

Your analysis of the reward structure and PD controller tracking errors is quite insightful! One small clarification: Diffuser actually uses receding horizon replanning rather than purely open-loop execution — it re-plans a fresh H-step trajectory from the current state every few steps.
Both Diffuser and AdaptDiffuser share the same H, but Diffuser's generated trajectories tend to be suboptimal — containing redundant detours within the maze. AdaptDiffuser's adaptation mechanism guides the diffusion model toward generating tighter, more direct paths that are closer to optimal. As a result, within the same H-step planning budget, AdaptDiffuser reaches the goal earlier and accumulates more reward — and this, rather than tracking error alone, is the primary driver of the score difference.

[L1U-C]

Thank you for your patient explanation！ It needs to be clarified that diffuser is special in maze2d environment, and it does use open-loop control, which may affect our analysis. Perhaps you have changed the manner to receding horizon control in maze2d?

[Liang-ZX]

Yes, the difference is just how to understand open-loop. Receding horizon planning is also defined as open-loop in Diffuser.

[L1U-C]

https://github.com/jannerm/diffuser/blob/13b4d404bdbd9d0d74c08d5a51ee39f84e765830/scripts/plan_maze2d.py#L51-L91
Maze2d does not seem to use Receding horizon planning, but only generates a trajectory once and generates actions according to the states of the trajectory. This does not seem to be consistent with the receding horizon control of gym-mujoco environment.
Thank you for your detailed analysis.

[Liang-ZX]

Oh, we have changed this part in AdaptDiffuser to also use receding horizon control

[L1U-C]

Thank you for your clarification.

However, due to the particularity of the maze2d environment, I still have one question regarding the horizon of the trajectory generated by the diffusion model under the receding horizon control.

Will the horizon of the trajectory decrease as the planning progresses? (Since the diffusion model generates the middle part of the trajectory under the constraints of the starting point and the ending point, as the planning progresses, will the starting point of the trajectory generated by the diffusion model remain the original starting point or change to the current planned position?)

Since the training of the trajectory diffusion model is conducted under a fixed horizon, although the diffuser allows configuring different horizons during planning, I would like to ask about your specific approach.

Thank you！

[Liang-ZX]

We just use a fixed planning horizon, but we only take the first step and then replan.

[L1U-C]

Is there a problem with the adoption of receding horizon control in maze2d?
For example, in the later stage of planning, the current position is very close to the end point, and whether it is inappropriate to use adaptdiffuser to generate a large number of points (number=horizon) in a very small distance.
Thank you!