3D Perspective Projection Matrix (Metal Part 11)

z4gon
z4gon

Setting up the the 3D perspective projection matrix to transform view space coordinates in homogeneous clip space coordinates. Later on the GPU takes in these and calculates the Normalized Device Coordinates, to finally calculate the actual Screen Space coordinates. Configuring the Depth Stencil in Metal, to perform Depth Testing and clipping based on depth, using the Depth Texture. Multiplying the projection matrix by the view space coordinates vector during the Vertex Shader Function stage.

Source Code

See Project in GitHub 👩‍💻

References


Table of Content


3D Perspective Projection Matrix

View Frustrum

Projecting 3D objects onto a flat surface means connecting each vertex to the position of the camera, and pin pointing where that line crosses the near clip plane.

Picture

Image Source 🔗

Clipping

The near and far clip planes determine what gets rendered in terms of depth.

The field of view means how much stuff gets into the projection in terms of bounds vertically and horizontally.

Picture

Image Source 🔗

Matrix

All the calculations can be put into a pre-determined matrix that can be multiplied by a position vector and it will return the homogenous clip space coordinates.

Metal will then translate these to Normalized Device Coordinates ranging from (-1, -1) to (1, 1) and including depth.

Finally this will be translated to screen space as x, y coordinates in pixels.

Picture

Image Source 🔗

This is what it looks like in code:

mutating func projectPerspective(fieldOfView: Float, aspectRatio: Float, farClippingDistance: Float, nearClippingDistance: Float) {
    var result = matrix_identity_float4x4

    let fov = fieldOfView
    let aspect = aspectRatio
    let far = farClippingDistance
    let near = nearClippingDistance

    result.columns = (
        float4(Float(1) / ( aspect * tan(fov / Float(2)) ), 0, 0, 0),
        float4(0, Float(2) / tan(fov / Float(2)), 0, 0),
        float4(0, 0, -((far + near)/(far - near)), -1),
        float4(0, 0, -((2 * far * near)/(far - near)), 0)
    )

    self = matrix_multiply(self, result)
}

Cube Mesh

To define a cube mesh, we need just 8 vertices.

And then an array of indexes to define all the counter clockwise triangles that will make up the quad faces.

class CubeMesh : Mesh{
    override func createMesh() {
        vertices = [
            Vertex(position: float3( 0.5, 0.5, 0.5), color: float4(1,0,0,1)), // FRONT Top Right
            Vertex(position: float3(-0.5, 0.5, 0.5), color: float4(0,1,0,1)), // FRONT Top Left
            Vertex(position: float3(-0.5,-0.5, 0.5), color: float4(0,0,1,1)), // FRONT Bottom Left
            Vertex(position: float3( 0.5,-0.5, 0.5), color: float4(1,0,1,1)),  // FRONT Bottom Right

            Vertex(position: float3( 0.5, 0.5, -0.5), color: float4(0,0,1,1)), // BACK Top Right
            Vertex(position: float3(-0.5, 0.5, -0.5), color: float4(1,0,1,1)), // BACK Top Left
            Vertex(position: float3(-0.5,-0.5, -0.5), color: float4(1,0,0,1)), // BACK Bottom Left
            Vertex(position: float3( 0.5,-0.5, -0.5), color: float4(0,1,0,1))  // BACK Bottom Right
        ]

        // counter clockwise mean out facing
        indices = [

            // FRONT face
            0,1,2,
            0,2,3,

            // BACK face
            5,4,7,
            5,7,6,

            // TOP face
            4,5,1,
            4,1,0,

            // BOTTOM face
            6,7,3,
            6,3,2,

            // LEFT face
            1,5,6,
            1,6,2,

            // RIGHT face
            4,0,3,
            4,3,7
        ]
    }
}

Depth Stencil

Descriptor

To let the GPU know what vertices are farther away from the camera, and then be able to clip them using the Depth Test, we need to setup the Depth Stencil Descriptor.

State

We will use the descriptor to create the Depth Stencil State, and pass it on to the Render Command Encoder.

public struct LessDepthStencilState: DepthStencilState {
    var name: String = "Less DepthTest"
    var depthStencilState: MTLDepthStencilState!

    init(){

        let depthStencilDescriptor = MTLDepthStencilDescriptor()
        depthStencilDescriptor.depthCompareFunction = MTLCompareFunction.less
        depthStencilDescriptor.isDepthWriteEnabled = true

        depthStencilState = Engine.device.makeDepthStencilState(descriptor: depthStencilDescriptor)
    }
}

Pixel Format

The Render Pipeline Descriptor and the MTKView will both need to set the Depth Stencil Pixel Format as well, just like they set the Color Pixel Format.

public struct BasicRenderPipelineDescriptor: RenderPipelineDescriptor{
    ...

    init(){

        ...

        // make the pixel format match the device
        renderPipelineDescriptor.colorAttachments[0].pixelFormat = Preferences.PixelFormat
        renderPipelineDescriptor.depthAttachmentPixelFormat = Preferences.DepthStencilPixelFormat

        ...
    }
}
class GameView: MTKView {

    var renderer: GameViewRenderer!

    required init(coder: NSCoder) {
        super.init(coder: coder)

        device = MTLCreateSystemDefaultDevice()
        clearColor = Preferences.ClearColor
        colorPixelFormat = Preferences.PixelFormat
        depthStencilPixelFormat = Preferences.DepthStencilPixelFormat

        Engine.initialize(device: device!)

        renderer = GameViewRenderer(self)
        delegate = renderer
    }
}

Mesh Renderer

The mesh renderer will pass the Depth Stencil State to the Render Command Encoder.

func doRender(renderCommandEncoder: MTLRenderCommandEncoder) {
    ...
    renderCommandEncoder.setDepthStencilState(DepthStencilStateCache.getDepthStencilState(.Less))
    ...
}

Camera

The camera will use its properties to calculate the Perspective Projection Matrix.

class Camera : Component, EarlyUpdatable {

    public var viewMatrix: float4x4 = matrix_identity_float4x4
    public var projectionMatrix: float4x4 = matrix_identity_float4x4

    public var type: CameraType = CameraType.Perspective

    public var fieldOfView: Float = 60
    public var nearClippingDistance: Float = 0.1
    public var farClippingDistance: Float = 1000

    ...

    func updateProjectionMatrix() {
        var result: float4x4 = matrix_identity_float4x4

        if(type == CameraType.Perspective) {
            result.projectPerspective(
                fieldOfViewDegrees: fieldOfView,
                aspectRatio: GameViewRenderer.AspectRatio,
                farClippingDistance: farClippingDistance,
                nearClippingDistance: nearClippingDistance
            )
        }

        projectionMatrix = result
    }

    // to ensure all other components get the accurate camera position
    func doEarlyUpdate(deltaTime: Float) {
        updateViewMatrix()
        updateProjectionMatrix()
    }
}

Scene

The Scene will include the projection matrix in its Scene Constants that will get passed down to the GPU.

class Scene : Transform {

    private var _sceneConstants: SceneConstants! = SceneConstants()

    ...

    func updateSceneConstants() {
        _sceneConstants.viewMatrix = CameraManager.mainCamera.viewMatrix
        _sceneConstants.projectionMatrix = CameraManager.mainCamera.projectionMatrix
    }

    override func render(renderCommandEncoder: MTLRenderCommandEncoder) {

        updateSceneConstants()

        // set the view matrix
        renderCommandEncoder.setVertexBytes(&_sceneConstants, length: SceneConstants.stride, index: 2)

        super.render(renderCommandEncoder: renderCommandEncoder)
    }
}

Shader

In the Shader code, we will use the projection matrix to multiply it by the rest of the matrices and the position, to complete the transformation of the coordinates into homogenous clip space.

float4 HCPosition = ProjMatrix * ViewMatrix * ModelMatrix * objectPosition;
struct SceneConstants {
    float4x4 viewMatrix;
    float4x4 projectionMatrix;
};

vertex FragmentData basic_vertex_shader(
  // metal can infer the data because we are describing it using the vertex descriptor
  const VertexData IN [[ stage_in ]],
  constant ModelConstants &modelConstants [[ buffer(1) ]],
  constant SceneConstants &sceneConstants [[ buffer(2) ]]
){
    FragmentData OUT;

    // return the vertex position in homogeneous screen space
    // ProjectionMatrix * ViewMatrix * ModelMatrix * ObjectPosition = HSCPosition
    OUT.position = sceneConstants.projectionMatrix
                    * sceneConstants.viewMatrix
                    * modelConstants.modelMatrix
                    * float4(IN.position, 1);

    OUT.color = IN.color;

    return OUT;
}

Result

Now the cube appears projected correctly in the screen space plane.

Picture