Touchscript for VR Interactions

[ludos1978]

Hi, i am thinking about using touchscript to interact with objects in VR. My project requires a lot of different input devices (Leap Motion, Vive Controllers, Mouse and Gaze). Vive and Leap are devices with multiple "touches".

I have already tried the demo's of touchscript with a VR headset, however when i the raycasts seem to be offset from the headset's position. I am unable to touch any of the objects or surfaces with vr glasses. This is depending on the setting of the "Target Eye" of the Camera. If i set None it works, any other setting breaks the input. Do you have a hint where i might start searching for the error?

thanks and good work!


(i previously posted this question on github, but closed it over there https://github.com/TouchScript/TouchScript/issues/233 )

[ludos1978]

i have found some insights to the problem. Camera.ScreenPointToRay does not work in VR. Instead ViewportPointToRay must be used to create a ray from the Camera. While this is a simple change in performWorldSearch it still does not work for "worldspace canvases". A drawray shows that it's now correct.

after the pixelRect.Contains(screenPosition) call add:
position = new Vector3(position.x / Screen.width, position.y / Screen.height, 0);

i'm looking into performUISearchForCanvas now...

[ludos1978]

i found a reason to stop researching. While touchscript is great for touch devices and other inputs i think there is no 3d source position of input devices possible. In Vr most input devices are not relative to the camera, but a device with a individual position in the room. Looking into the source code looked to me that it will be hard introducing 3d source positions for inputs.

now i will stop monologizeing

[valyard]

Hi.

I actually was thinking about making a VR branch for TouchScript. You are definitely correct, there has to be some sort of 3D Layer Volume and an ability to use 3D rays and 3D points as inputs, i.e. Leap Motion fingers (3D points) or Vive Controllers (Rays). This is why I am making Touch.Position to be Vector3.

There's a lot to think about, but I really want to move this to VR. Do you want to discuss this project?