Is Enable camera pose optimization #83 solved？

[TwiceMao]

Is Enable camera pose optimization #83 solved？
Thx!

[wanmeihuali]

The recent finding is that: even for pose optimization only(Given a trained scene), some pose still does not converge. I'm still debugging it.

[ttsesm]

Hi guys,
What do you mean by "enable camera pose optimization"?

Do you mean to solve for the camera pose instead of the image during the inference time or something else?

Thanks.

[wanmeihuali]

@ttsesm Yep, as long as the training process is

$$ loss=L(render(gaussian, T_{camera})) $$

$$ gaussian=gaussian-\delta_{lr} \frac{\partial{loss}}{\partial{gaussian}} $$

It shall also be possible to optimize the camera pose:

$$ T_{camera}=T_{camera}-\delta_{lr} \frac{\partial{loss}}{\partial{T_{camera}}} $$

Given a trained scene and a camera pose with noise, we shall be able to optimize the camera pose to remove noise from it. Further, given a list of image-camera pose pairs in which the camera pose is noisy, we might still be able to train the scene.

The following papers may help you learn more about pose optimization in NeRF:
barf: https://arxiv.org/abs/2104.06405
nope-nerf: https://arxiv.org/abs/2212.07388

[ptg666]

Hi, has the problem of camera pose optimization been solved? @wanmeihuali

[QitaoZhao]

Same question here!