Backface culling in gfx

2018-06-02 ·

So I’m trying to render a few cubes.

Hmm, something doesn't look right.

Now something doesn’t look quite right here. I’ve probably got the winding order wrong somewhere, but apart from anything else, it looks like we’re not culling backfaces yet.

For those not familiar with the term, a “backface” is just the reverse side of a polygon. We should never see them because they should be on the inside of our model, so we use backface culling to avoid rendering them, for efficiency. Failing to do this, and having your models be inside-out, has some strange effects which you can see on the bottom row of cubes there.

So how do you cull backfaces in gfx? A quick search doesn’t turn up much, so let’s turn to documentation: https://docs.rs/gfx/0.17.1/gfx/?search=cull

Search results, including CullFace and with_cull_back

The with_cull_back function looks promising, but I’m not sure what a Rasterizer is or where to use one. So instead, I’m just going to grep through gfx-rs’s code and see if I can find this function used anywhere.

It looks like there is one place in the shadow example that calls it. There’s a lot going on there that I don’t understand though - the create_pipeline_state function doesn’t look as simple as the create_pipeline_simple function, for some reason.

Back to the docs!

So the first thing I notice is that create_pipeline_from_program exists, and would allow me to avoid having to think about the ShaderSet parameter, so I’ll try that first.

    // Old PSO - without backface culling:
    // let pso = factory
    //     .create_pipeline_simple(
    //         vertex_source.as_bytes(),
    //         fragment_source.as_bytes(),
    //         pipe::new()
    //     ).unwrap();

    let pso = {
        let program = (); // ???
        let rasterizer = (); // ???

        factory
            .create_pipeline_from_program(
                &program,
                Primitive::TriangleList,
                rasterizer,
                pipe::new())
            .unwrap()
    };

This is mostly straightforward. The main differences are that I have to make my own shader program (instead of just passing the raw shader source), and I have to supply a rasterizer.

That second part is easy, I can probably just copy it from the gfx example:

        let rasterizer = Rasterizer::new_fill()
            .with_cull_back();

(I left out the with_offset part because that’s not really relevant to ordinary rendering. That example is probably using it to avoid shadowing artefacts.)

Next up is the shader program. Searching the docs again suggests both create_program and link_program. The first one sounds like what I want, but it expects a ShaderSet. The second only asks for shader source, which is closer to what I’m using right now.

        let program = factory.link_program(
            vertex_source.as_bytes(),
            fragment_source.as_bytes(),
        ).unwrap();

That compiles, runs, and gives us:

The cubes are now rendering correctly!

Oh, cool. I guess the cubes were fine then? Honestly, I’d expected to have to fix up the mesh once I got culling working. The fact that the backfaces were showing through even though all of the polygons were facing the correct way makes me think that maybe the depth buffer isn’t being used.

But hey, I’ll cross that bridge when I come to it.

Fixing the depth buffer: an edit

2018-06-05

Getting depth working was simple enough that I didn’t think it warranted its own post. It only required three small changes.

First, add a depth test definition to the pipeline:

    pipeline pipe {
        vbuf: VertexBuffer<Vertex> = (),
        transform: ConstantBuffer<Transform> = "Transform",
        scissor: Scissor = (),
        depth: DepthTarget<DepthFormat> = LESS_EQUAL_WRITE, // This bit
        out: RenderTarget<ColorFormat> = "target",
    }

The LESS_EQUAL_WRITE means that we only render a pixel if its depth is less than or equal to the current depth at that point - and if so, we also write to the depth buffer. (You could also use LESS_EQUAL_TEST, which is the same but does not write to the depth buffer. This would be useful for transparent objects that have been pre-sorted.)

Next, we need to reference the depth target in our pipeline data:

    let mut quad_data = pipe::Data {
        vbuf: quad_buffer,
        transform: transform_buffer.clone(),
        scissor: Rect { x: 0, y: 0, w: 0, h: 0 },
        depth: depth_view.clone(),  // This bit
        out: color_view.clone(),
    };

I cloned it because I’m also passing it into other data structs. As far as I’m aware, it’s an Arc so shouldn’t incur much of a cost to do so.

Finally, we need to clear the depth buffer every frame:

    encoder.clear_depth(&depth_view, 1.0);

A value of 1.0 is the maximum depth, meaning “as far away as possible”. Anything closer (e.g. everything) will be rendered, hence why this is considered a “clear” depth buffer.

Here’s what it looked like before and after:

Before and after depth testing