Picking tutorial, intersecting a mesh with a mouse ray
By Rim van Wersch, April 9 2006
Intersecting a mesh with a mouse ray, often referred to as picking, is a technique to select meshes in your 3D world by clicking your mouse. In this tutorial we'll take a look at a nice clean way to do this in MDX and what can be done with the information obtained from an intersection test. This tutorial comes with a port of the C++ picking sample from the SDK.
How does it work?
The technique of picking essentially comes down to translating your mouse coordinates to a ray defined in world space and performing an intersection test between this ray and suitable meshes you want to be able to pick. Suitable meshes typically means visible meshes, so as a side note, picking may be optimized by using view frustum checks and/or occlusion queries to reduce the number of intersection tests. But let's not get ahead of ourselves and preview the steps we'll perform in this tutorial.
- Obtain proper mouse coordinates
- Construct a world space ray from these screen spaced coordinates
- Perform the intersection test between the ray and a mesh, ie. performing the actual pick
- Use intersection information to obtain the intersected triangle, which may be useful in some applications
Now that we have our mouse coordinates, let's take a look at what's going on in our DoPicking method, for which you can find the code below. The first thing you'll notice is that we clamp our mouse coordinates to our device's viewport. For translating our mouse coordinates to a world space ray, we'll be using our viewport's height and width to do some calculations (as explained in Toymaker's tutorial linked below), so we'll need to make sure the coordinates are contained within this expected range. Normally you wouldn't need to worry about this, but since we're capturing mouse movements (tracking drags outside our control's client area), we'll need this adjustment.
Next we move on to constructing the ray. We define two Vector3's that describe the origin (near) and direction (far) of the ray in screen space. By using the Vector3.Unproject method, we translate this screen space ray into our mesh's model space. What's that you say? We were going to translate the ray into world space? That's correct, but for performing the intersection test, we'll need to have our ray define in the mesh's model space, so we conveniently supply the world matrix used for our mesh to the Unproject method to skip this final transformation. By supplying Matrix.Identity you could obtain the ray defined in world space, but for typical intersection tests you won't need this.
Now that we finally have obtained our mouse ray in the desired model space, we can perform the intersection test. The Mesh.Intersect method provides various overloads that allow you to retrieve all intersections of the ray with the mesh, but since these can be handled exactly like the closest intersection, we'll stick to this closest one only for clarity. We obtain the IntersectionInformation object from the Mesh.Intersect method as an out parameter, while the boolean returned from this function indicates whether or not an intersection occured. For most applications, you'll only want to know if a specific mesh was clicked on, so in these cases the boolean value is sufficient and you're done.
If you need further information about the intersection, you can use the IntersectInformation object to calculate what you need. This may be useful for a modelling package for example, where users need to be able to select a specific face. So, let's investigate how we can obtain this face from our mesh by using the data in our IntersectionInformation object.
The IntersectionInformation object contains the index of the face that was intersected during the test, so we'll need to find a way to extract the vertex information for this face from our mesh. Fortunately, that's not too hard, as you can see in the code for this below. Since all faces in a typical D3DX mesh are triangles, we can easily obtain the indices for the intersected triangle from the mesh's index buffer by reading three indices starting from 3 * FaceIndex. Once we have these indices, it's simply a matter of extracting the vertex information from the mesh's vertex buffer for these indices.
This concludes our tutorial on performing picking. One step remains undiscussed here, that of determining the exact point of intersection by using the barycentric coordinates supplied in our IntersectionInformation object. This is however rarely used for picking, so if you require more information on this, please refer to the article on FlipCode linked to below or to the SDK documentation, where you'll find this explained in more detail.
About the sample project
The sample project you can download below is a port of the C++ picking sample provided in the DirectX SDK, with the ommission of checking for multiple intersections and calculating the exact point of intersection using the barycentric coordinates. For more information about the latter, please refer to the link below to the article on FlipCode. Handling multiple intersections isn't much different from handling a single one, so this is left as an exercise for the reader
The transformation of the mouse coordinates from screen space to world space is implemented a bit easier by using the Vector3.Unproject function from the Managed DirectX API. Under the hood this works exactly like the C++ sample and as such uses the same steps as explained in Toymaker's turorial on picking, which you can also find in the Further Reading section below.
Files for this tutorial