NA-MIC Project Weeks

Back to Projects List

SlicerVR - Restore Interactions

Key Investigators

• Csaba Pintér (EBATINCA, Spain)
• Simon Drouin (ÉTS Montréal, Canada)
• Jean-Christophe Fillion-Robin (Kitware, Inc., USA)
• Andrey Titov (ÉTS Montréal, Canada)
• Tina Nantenaina (ÉTS Montreal, Canada)
• Lea Vong (ÉTS Montréal, Canada)
• Lucas Gandel (Kitware, Inc., France)

Presenter location: Remote

Project Description

The main controller interactions in SlicerVR have been broken for about a year, some interaction types even longer. It would be crucial for keeping SlicerVR usable to make the interactions work again.

Kitware and Robarts (Jean-Christophe Fillion Robin, Lucas Gandel, Sankhesh Jhaveri, Adam Rankin) have been investing resources and effort in rehauling the AR/VR backend in VTK for a while, thus now we have a new OpenXR backend and restructured libraries SlicerVR is built on. The goal is to give a small push to their efforts in terms of SlicerVR interactions during the project week, towards restoring at least the previous feature set.

Objective

In PW 37, basic interaction has been fixed.

1. Fix the two-controller world move/zoom (i.e. 3D pinch)
2. Customization of controller buttons. Either via the
   • Method in-place (functions integrated here, see also here)
   • Json manifest files (see here)

Approach and Plan

1. Set up a VR workstations at ETS to be able to test and develop
2. Fix two hands interactions
3. Implement custom interaction to test both customization methods

Progress and Next Steps

1. Fixed 3D pinch interaction in commit SlicerVirtualReality@49f1896d6
2. Rebased the GUI widgets branch into a new branch
3. Fixed lookup of vtk_openvr_actions.json and vtk_openvr_binding_*.json files for both build and install tree. See commit SlicerVirtualReality@a4d465b73 integrated through PR-117
4. Make in-VR GUI widget work (with many workarounds and limitations)

Illustrations

Laser pointer interactions with widget existing in Slicer:

Laser pointer interactions with custom widget:

More comprehensive demonstration:

Background and References

Past project week pages

• Project week #38 page
• Project week #37 page
• Project week #35 page
• Project week #34 page

Pinter, C., Lasso, A., Choueib, S., Asselin, M., Fillion-Robin, J. C., Vimort, J. B., Martin, K., Jolley, M. A. & Fichtinger, G. (2020). SlicerVR for Medical Intervention Training and Planning in Immersive Virtual Reality. IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 2, pp. 108-117, May 2020, doi: 10.1109/TMRB.2020.2983199.