Introduction

Welcome! This book will guide you through the creation and design of a 3D renderer using the Rust programming language and the Vulkan graphics API.

Purpose

As of January 2021, the resources for learning Vulkan are scarce. The existing resources largely focus on C++ and cover various rendering techniques. This is excellent! However, the focus for many of these resources is not on building a structured program that goes beyond the scope of tutorial code. This book is meant to be higher level, demonstrating how to build a 3D world and render it in realtime. This will be particularly of use to indie game developers looking to create a 3D game from scratch without getting overwhelmed.

Why Rust?

Rust is a great alternative to C++! A few of the benefits:

  • It provides a smooth workflow for developers with clear, specific error messages from the compiler
  • rustup and cargo make managing rust toolchain installations and rust projects straightforward
  • The lints from clippy help to improve the code quality and catch certain common mistakes
  • rustfmt handles formatting the code and code style
  • Memory safety. Code written outside of unsafe blocks is checked by the borrow checker

Target Audience

The target audience of this book is moderately experienced developers with an interest in graphics programming. Prior graphics programming experience, particularly with OpenGL will be particularly useful. This is the book I would have wanted to read when first starting out with Vulkan.

What Is Covered

This book is very code-heavy and implementation focused, as opposed to other resources that may be more focused on theory.

What Is Not Covered

This book will not go into detail on linear algebra concepts or mathematics, as there are already great resources available. This book may not go into as much depth on particular parts of the Vulkan API as other resources might. A list of useful external resources for building upon the content of this book can be found in the Further Reading section of the appendix.

Project Repo

The source code for the Obsidian render built in this book can be found on github:

https://github.com/matthewjberger/obsidian

The source code for this mdbook can also be found on github:

https://github.com/matthewjberger/letsbuildarenderer

Final Project Preview

Chapter 1

  • Set up the project structure
  • Get a window up and running
  • Create the boilerplate code used throughout the rest of the book

The code in this chapter will serve as the project's foundation.

file-structure

Getting Started

Dependencies

This project requires the Rust programming language.

The easiest way to get access to the Vulkan dependencies is to install the Vulkan SDK. This provides access to the Vulkan Configurator, glslangvalidator, the debug layers, and other useful tools for working with Vulkan.

The latest version of GCC and CMake will also be required for ffi bindings to C/C++ libraries.

Development Environment Setup

Using vscode and rust-analyzer with the rust-analyzer vscode extension is recommended. However, any Rust development environment you are comfortable with will work.

The official Rust book is a great resource if you are new to Rust.

Quick Windows Setup

On windows, installing programs can be trickier than on other platforms. It is recommended to use a package manager such as Scoop or chocolatey.

First, make sure PowerShell 5 (or later, include PowerShell Core) and .NET Framework 4.5 (or later) are installed. Then run:

# Install scoop
Set-ExecutionPolicy RemoteSigned -scope CurrentUser
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh')

# Scoop uses git to update itself and 7zip to extract archives
scoop install git 7zip 

# Install the project's dependencies
scoop install gcc cmake rustup

# Set the stable rust toolchain as the default toolchain
rustup default stable

# Install vscode, kept in a separate bucket called the 'extras' bucket
scoop bucket add extras
scoop install vscode

Project Setup

The directory structure for the project will look like this:

file-structure

File Structure

  • assets/
    • hdr/ Contains HDR maps used for skyboxes and environment mapping
    • models/ Contains 3D assets, such as *.gltf and *.glb files
    • shaders/ Contains all of the shaders used in the project
  • crates/
    • obsidian_app/ A library that handles the window and application input boilerplate
  • viewer/ An application that can render 3D models, developed over the course of this book

Creating the File Structure

Create the project as a library:

cargo new --lib obsidian
cd obsidian

Create the asset folders:

mkdir assets
mkdir assets/hdr
mkdir assets/icon
mkdir assets/models
mkdir assets/shaders

Edit Cargo.toml to make the project a cargo workspace:

# Leave the [package] section as is

[workspace]
members = ["crates/*", "viewer"]
default-members = ["viewer"]

[dependencies]
obsidian_app = { path = "crates/obsidian_app" }

Create the application crate as a library:

cargo new --lib crates/obsidian_app

Create and edit crates/obsidian_app/src/app.rs to add an Application struct. This will eventually contain everything needed for an Obsidian application, including a physics world, an entity component system, the renderer itself, and more.

pub struct Application;

Update crates/obsidian_app/src/lib.rs:

mod app;
pub use self::app::*;

Edit src/lib.rs to reference the app library:

pub mod app {
    pub use obsidian_app::*;
}

Create the viewer application:

cargo new viewer

Update the viewer/cargo.toml to reference all of obsidian as a single library.

[dependencies]
obsidian = { path = ".." }

Now, run cargo run --release to compile and statically link the program. You should see:

"Hello, world!"

Creating the Application

To make our application boilerplate reusable, we created the obsidian_app library. Now, we can start filling it out!

Configuring the Application

Add an application configuration struct:

pub struct AppConfig {
    pub width: u32,
    pub height: u32,
    pub title: String,
}

impl Default for AppConfig {
    fn default() -> Self {
        Self {
            width: 800,
            height: 600,
            title: "Obsidian Application".to_string(),
        }
    }
}

Dependencies

We will need to pull in crates for windowing, error handling, and the rust logger facade.

Add the following to crates/obsidian_app/Cargo.toml:

anyhow = "1.0.34"
log = "0.4.11"
winit = "0.24.0"

And add the following to viewer/Cargo.toml:

anyhow = "1.0.34"
log = "0.4.11"

Generalizing the Application

The application can be broken down into discrete steps, such as initialization, updating, handling events, and more. This can be described with a trait:

pub trait Run {
    fn initialize(&mut self, _application: &mut Application) -> Result<()> {
        Ok(())
    }

    fn update(&mut self, _application: &mut Application) -> Result<()> {
        Ok(())
    }
}

Now we can create a function to execute any type that implements the trait:

pub fn run_application(mut runner: impl Run + 'static, configuration: AppConfig) -> Result<()> {
    // TODO ...
}

The Event Loop

Now we can fill out the body of the run_application function.

We can create the window, an instance of the application, and call the initialize function of the runner.

    let (event_loop, _window) = create_window(&configuration)?;

    let mut application = Application {};

    log::info!("Running Application");
    runner.initialize(&mut application)?;

The winit crate manages the event loop. For easy error handling, we will use a lambda that returns a Result<()> to handle each cycle. This lets us check for application errors and log them.

Inside of the event loop we can also handle events and invoke the application runner's methods.

Note: The control_flow object allows controlling the main loop's behavior. For a game loop or realtime rendering application we will need to execute the loop continously, so we use ControlFlow::Poll.

    event_loop.run(move |event, _, control_flow| {
        let mut cycle_result = || -> Result<()> {
            *control_flow = ControlFlow::Poll;
            match event {
                Event::MainEventsCleared => {
                    runner.update(&mut application)?;
                }
                Event::WindowEvent {
                    event: WindowEvent::CloseRequested,
                    ..
                } => {
                    *control_flow = ControlFlow::Exit;
                }
                Event::LoopDestroyed => {
                    info!("Exited application");
                }
                _ => {}
            }
            Ok(())
        };
        if let Err(error) = cycle_result() {
            error!("Application Error: {}", error);
        }
    });

Source Code

The full source for the app module code presented so far look like this:

// crates/obsidian_app/src/app.rs
use anyhow::Result;
use log::{error, info};
use winit::{
    dpi::PhysicalSize,
    event::{Event, WindowEvent},
    event_loop::{ControlFlow, EventLoop},
    window::{Window, WindowBuilder},
};

pub struct AppConfig {
    pub width: u32,
    pub height: u32,
    pub title: String,
    pub logfile_name: String,
}

pub fn create_window(config: &AppConfig) -> Result<(EventLoop<()>, Window)> {
    let event_loop = EventLoop::new();

    let window = WindowBuilder::new()
        .with_title(config.title.to_string())
        .with_inner_size(PhysicalSize::new(config.width, config.height))
        .build(&event_loop)?;

    Ok((event_loop, window))
}

impl Default for AppConfig {
    fn default() -> Self {
        Self {
            width: 800,
            height: 600,
            title: "Obsidian Application".to_string(),
            logfile_name: "obsidian.log".to_string(),
        }
    }
}

pub struct Application;

pub trait Run {
    fn initialize(&mut self, _application: &mut Application) -> Result<()> {
        Ok(())
    }

    fn update(&mut self, _application: &mut Application) -> Result<()> {
        Ok(())
    }
}

pub fn run_application(mut runner: impl Run + 'static, configuration: AppConfig) -> Result<()> {
    let (event_loop, _window) = create_window(&configuration)?;

    let mut application = Application {};

    log::info!("Running Application");
    runner.initialize(&mut application)?;

    event_loop.run(move |event, _, control_flow| {
        let mut cycle_result = || -> Result<()> {
            *control_flow = ControlFlow::Poll;
            match event {
                Event::MainEventsCleared => {
                    runner.update(&mut application)?;
                }
                Event::WindowEvent {
                    event: WindowEvent::CloseRequested,
                    ..
                } => {
                    *control_flow = ControlFlow::Exit;
                }
                Event::LoopDestroyed => {
                    info!("Exited application");
                }
                _ => {}
            }
            Ok(())
        };
        if let Err(error) = cycle_result() {
            error!("Application Error: {}", error);
        }
    });
}

Creating the Viewer

Finally, we can use our boilerplate code to create the Viewer.

// obsidian/viewer/src/main.rs
use anyhow::Result;
use log::info;
use obsidian::app::{run_application, AppConfig, Application, Run};

pub struct Viewer;

impl Run for Viewer {
    fn initialize(&mut self, _application: &mut Application) -> Result<()> {
        info!("Viewer initialized");
        Ok(())
    }

    fn update(&mut self, _application: &mut Application) -> Result<()> {
        Ok(())
    }
}

fn main() -> Result<()> {
    let viewer = Viewer {};
    run_application(
        viewer,
        AppConfig {
            title: "Obsidian Viewer".to_string(),
            logfile_name: "viewer.log".to_string(),
            ..Default::default()
        },
    )
}

Now, when you run the application with cargo run --release from the project root the application will display an empty window with a custom title!

file-structure

The Logging Backend

In order for the logger facade functions to work, we will need to add a logging backend. For this project, we'll use simplelog.

Add this dependency to crates/obsidian_app/Cargo.toml:

simplelog = { version = "0.9.0", features = ["termcolor"] }

Note: The termcolor feature allows for colored terminal log output.

Now we can create a logger module to setup the logger backend.

// crates/obsidian_app/src/lib.rs
mod logger;
...

// crates/obsidian_app/src/logger.rs
use anyhow::{Context, Result};
use simplelog::{CombinedLogger, Config, LevelFilter, TermLogger, TerminalMode, WriteLogger};
use std::{fs::File, path::Path};

pub fn create_logger(path: impl AsRef<Path>) -> Result<()> {
    let name = path.as_ref().display().to_string();
    let error_message = format!("Failed to create log file named: {}", name);
    let file = File::create(path).context(error_message)?;
    CombinedLogger::init(vec![
        TermLogger::new(LevelFilter::Info, Config::default(), TerminalMode::Mixed),
        WriteLogger::new(LevelFilter::max(), Config::default(), file),
    ])?;
    Ok(())
}

The CombinedLogger lets us create a TermLogger and a WriteLogger at the same time. We will only log messages with the severity Info and above to the terminal, and we will log all messages to the configuration file.

And finally we can invoke the create_logger function in our run_application method:

pub fn run_application(mut runner: impl Run + 'static, configuration: AppConfig) -> Result<()> {
    create_logger(&configuration.logfile_name)?;
    ...
}

Creating the Rendering Library

To keep the rendering code separated from the rest of the codebase, we will create a new library called obsidian_render.

Adding the Render Crate

To start, we can create a crate for handling graphics.

cargo new --lib crates/obsidian_render

Then we can link obsidian against the obsidian_render library, by listing it as a dependency in obsidian/Cargo.toml

[dependencies]
...
obsidian_render = { path = "crates/obsidian_render" }

and expose it as a library module.

// src/lib.rs

...

pub mod render {
    pub use obsidian_render::*;
}

Dependencies

Add dependencies for error handling, logging, and a new dependency named raw-window-handle to crates/obsidian_render/Cargo.toml.

anyhow = "1.0.34"
log = "0.4.11"
raw-window-handle = "0.3.3"

raw-window-handle is a library that abstracts platform specific window handles. The winit library uses the abstraction from this library to provide a window handle via the HasRawWindowHandle trait implementation on the Window type.

Render Trait

To prevent coupling any one specific backend to the rest of the application, the render library can expose the renderer via a Render interface and consumers of the library can request a specific backend via a Backend enum. The Render interface is not intended to be a low level common abstraction over various graphics API's, but rather a high level abstraction meant to render the 3D world we will be creating later in this book.

There is an excellent cross-platform graphics and compute abstraction library in the rust ecosystem named gfx-rs.

The Render Module

Create a new module named render.rs

touch crates/obsidian_render/src/render.rs

Update the crates/obsidian_render/src/lib.rs to list the new module:

pub mod render;
pub use crate::render::{Backend, Render};

We can list our graphics backends with an enum:

pub enum Backend {
    Vulkan,
}

The Render trait can be written as:

pub trait Render {
    fn render(
        &mut self,
        dimensions: &[u32; 2],
    ) -> Result<()>;
}

The render call will eventually be given a parameter containing a description of our World to render. The World implementation will come in a later chapter.

Vulkan Backend

Now that the Render trait exists, we will need to create a backend that implements it.

Render Backend Feature Flags

To allow compiling a specific backend, we will use feature flags. For the purpose of this book, we will only be implementing the Vulkan backend so it will be a default feature.

# crates/obsidian_render/Cargo.toml
[features]
default = ["vulkan"]
vulkan = []

Vulkan Render Backend

Setting Up the Backend

Create a new module named vulkan.rs and a folder for its modules:

touch crates/obsidian_render/src/vulkan.rs
mkdir crates/obsidian_render/src/vulkan

Update the crates/obsidian_render/src/lib.rs to list the new module:

#[cfg(feature = "vulkan")]
mod vulkan;

Creating the Vulkan Render Module

Create a file for the Vulkan specific render module:

touch crates/obsidian_render/src/vulkan/render.rs

Declare it as a module, and expose the VulkanRenderBackend to the crate:

pub(crate) use self::render::VulkanRenderBackend;

mod render;

Declare the VulkanRenderBackend as a plain struct that implements the Render trait:

// crates/obsidian_render/src/vulkan/render.rs
use crate::Render;
use anyhow::Result;
use raw_window_handle::HasRawWindowHandle;
use log::info;

pub(crate) struct VulkanRenderBackend;

impl Render for VulkanRenderBackend {
    fn render(
        &mut self,
        _dimensions: &[u32; 2],
    ) -> Result<()> {
        Ok(())
    }
}

impl VulkanRenderBackend {
    pub fn new(
        _window_handle: &impl HasRawWindowHandle,
        _dimensions: &[u32; 2],
    ) -> Result<Self> {
        info!("Created Vulkan render backend");
        Ok(Self{})
    }
}

Instantiating Graphics Backends

We can now write an associated method for the Render trait to provide a trait object (some type implementing the Render trait) by specifying the desired backend.

// creates/obsidian_render/src/render.rs
#[cfg(feature = "vulkan")]
use crate::vulkan::VulkanRenderBackend;

impl dyn Render {
    pub fn create_backend(
        backend: &Backend,
        window_handle: &impl HasRawWindowHandle,
        dimensions: &[u32; 2],
    ) -> Result<impl Render> {
        match backend {
            Backend::Vulkan => VulkanRenderBackend::new(window_handle, dimensions),
        }
    }
}

Using the Renderer

Now we can instantiate a renderer in our obsidian_app library, before moving on to filling out the Vulkan graphics backend.

List the obsidian_render library as a dependency in crates/obsidian_app/Cargo.toml:

[dependencies]
...
obsidian_render = { path = "../obsidian_render" }

Now we can instantiate the renderer by modifying crates/obsidian_app/src/app.rs.

Import the library types:

use obsidian_render::{Render, Backend};

Add a renderer property to the Application struct as a boxed trait object, and instantiate it in a constructor:

pub struct Application {
    pub renderer: Box<dyn Render>,
}

impl Application {
    pub fn new(window: &Window) -> Result<Self> {
        let logical_size = window.inner_size();
        let window_dimensions = [logical_size.width, logical_size.height];
        let renderer = Box::new(Render::create_backend(
            &Backend::Vulkan,
            window,
            &window_dimensions,
        )?);
        Ok(Self { renderer })
    }
}

The Application can now be created with the constructor in the run_application method:

// '_window' becomes 'window' here since it is now used
let (event_loop, window) = create_window(&configuration)?;

let mut application = Application::new(&window)?;

If you run the program now with cargo run --release, the log output should include a message saying that the Vulkan render backend was created.

Vulkan Context

Now we can start using Vulkan! This section will detail a self contained structure for setting up and store the fundamental objects required to run a Vulkan application.

From this point on in the book, the instructions for setting up and creating the files will not detail each line and step, and will rely on the accompanying source code for reference. All important sections of code will be explained, however!

The structure will be called a Vulkan Context. This is not an official Vulkan term, but rather the name for our grouping of Vulkan objects.

pub struct Context {
    pub allocator: Arc<vk_mem::Allocator>,
    pub device: Arc<Device>,
    pub physical_device: PhysicalDevice,
    pub surface: Option<Surface>,
    pub instance: Instance,
    pub entry: ash::Entry,
}

The order the struct fields are declared in determines the order that they are Dropped in. This will become important later, as cleaning up Vulkan resources has to happen in a particular order. Resources must not be in use when they are destructed, so declaring the struct this way enforces the correct order.

In reverse order, the fields are as follows.

  • entry

    The function loader from the ash library.

  • instance

    A wrapper around a vk::Instance, which stores application state because there is no global state in Vulkan.

  • surface

    A wrapper around a ash::extension::khr::Surface. A surface is required when rendering to a window.

  • physical_device

    A wrapper around a vk::PhysicalDevice. On simple systems, this a physical device represents a specific, physical GPU.

  • device

    A wrapper around a vk::Device. A logical device represents the application's view of the physical device. Vulkan calls will be made primarily on this object.

  • allocator

    Vulkan requires the application to handle allocating memory on its own. Thankfully, AMD has released a library that does this task well, called the Vulkan Memory Allocator. The rust bindings are provided by vk-mem-rs. Using a type from the vk-mem-rs library, we can create and store a memory allocator.

Vulkan Handles

Vulkan objects are constructed via API calls, and return handles. The wrappers we will create instantiate particular Vulkan objects, store their handles, and implement the Drop trait to call a destroy_instance API call which frees the resource. This intentionally ties the lifetime of the Vulkan object to lifetime of the Rust wrapper type.

Vulkan Instance

The first wrapper needed to create our Vulkan context will be the Instance. This is a wrapper around the ash::Instance type.

pub struct Instance {
    pub handle: ash::Instance,
}

The constructor on this wrapper is as follows.

pub fn new(entry: &ash::Entry, extensions: &[*const i8], layers: &[*const i8]) -> Result<Self> {
    let application_create_info = Self::application_create_info()?;
    Self::check_layers_supported(entry, &layers)?;

    let instance_create_info = vk::InstanceCreateInfo::builder()
        .application_info(&application_create_info)
        .enabled_extension_names(extensions)
        .enabled_layer_names(layers);

    let handle = unsafe { entry.create_instance(&instance_create_info, None) }?;
    Ok(Self { handle })
}

Vulkan Surface

Vulkan Physical Device

Vulkan Logical Device

Appendix

A - Further Reading

  • The Vulkan Tutorial by Alexander Overvoorde

    This is where most people learn Vulkan. It goes over all the fundamentals of Vulkan and focuses on explaining the API and using it to make a graphics application.

  • A Rust Version of the Vulkan Tutorial

    This is an excellent Rust port of the C++ Vulkan Tutorial.

  • Sascha Willems Vulkan Tutorials

    Sascha Willems has a github repo full of a variety of examples of how to use the Vulkan API to achieve various graphics programming techniques. This is highly recommended for anyone using the Vulkan API.

  • Learn OpenGL

    This is a website about graphics programming that focuses on OpenGL programming, but the techniques are all applicable to Vulkan.

B - Keywords

C - Vulkan Configurator