OpenGL

Notes and reminders aroud my understanding of modern OpenGL (4.2+)

A program (the client) issues commands, and these commands are interpreted and processed by the GL (the server).

TODOs

  • Find a spec definition of Command Stream

  • Find a spec defintion of what it means for a command to “reach the GL server”. (In particular, does it mean it is actually on the GPU, or could it just reach an internal driver buffer in main memory?)

  • I’d like to get a formal definition of what the spec exactly means by:
    • vertex array (e.g. p362 sec 10.3.9)

      Additionally, each vertex array has an associated binding so there is a buffer object binding for each of the vertex attribute arrays. The initial values for all buffer object bindings is zero. (p84 bottom)

    • vertex attribute array e.g. first line p365

      “If an enabled vertex attribute array is instanced”

  • Understand derived state:

    Some of [the state] […] is derived state visible only by the effect it has on how OpenGL operates. (p3 top)

  • Complete the list of fixed-function stages

  • Complete compute rendering command section

Conventions

“GLSL matrices are always column-major” (https://www.khronos.org/opengl/wiki/Interface_Block_(GLSL))

Shader code use pre-multiplication: tansformation * vector.

Prefixes in the C language:

  • functions: gl
  • constants: GL_
  • types: GL (e.g. GLfloat)

Data flow

Figure 3.1: Block diagram of the GL pipeline

Execution model

The application (client) issue commands executed by the GL (server). Commands are always processed in the order they are received.

Implementation can buffer commands in a command queue.

Context and Object model

A server may maintain a number of GL contexts, each of which is an encapsulation of current GL state and objects. A client may choose to be made current to any one of these contexts.

p3 1.2.2:

OpenGL contains […] many types of objects representing programmable shaders and the data they consume and generate, as well as other context state controlling non-programmable aspects of OpenGL.

Context state

Context state belongs to the GL context as a whole, rather than to specific instances of GL objects. It controls fixed-function stages of the GPU and specifies bindings of objects to the context. There are two types server state (majority) and client state. (p27 2.5)

Fixed function stages:

  • Clipping
  • Rasterization
  • Framebuffer clear
  • (TODO)

Generalities on objects

OpenGL API gives access to objects, which are identified by their name, Each type of object has a distinct name space, where names are a GLuint value (not actual text).

Names are distinct from the underlying objects: glGen*() functions reserve and return previously unused names, but the objects are created and bound to names later, through other GL functions (e.g. when names are bound to the context with glBind*()). Objects are modeled as a state vector, creating the object notably creates this state vector.

GL defines many types of objects, applications operate on instances of those objects. Specific instances are bound to a context.

Binding objects to the context determines which objects are used during commands execution.

“Standard” OpenGL Objects operations

The object model describes how most types of objects are managed by the application.

Available names are obtained and reserved with glGen*(). Underlying object is created and associated to a given name with first call to glBind*() for this name: it creates a state vector for the object with default values by the spec. Additionnaly, the bound object is the one affected by further GL operations/queries for this type. Some type of objects have a target (such as Buffer Objects, in which case there is one distinct binding point for each target.

Object might be created and associated to a new name at once with glCreate*().

Names are marked as unused with glDelete*(), but the underlying object and its state are not deallocated until it is not longer in use.

Query commands start with glGet.

Event Model

Fence

A Fence Object is associates with a Fence Command placed in the GL command stream. The Fence Object is initially UNSIGNALED, and becomes SIGNALED at some point after the Fence Command is completed. Fence command completion guarantees all preceding commands in the same command stream did complete.

Queries

Query Objects give access to asynchronous queries: a mechanism to return information about the processing of a sequence of GL commands.

Query object names are reserved with glGenQueries(). A query object is associated to the name with glBeginQuery*() (or glQueryCounter() for timer query), for provided target. The state is queried via glGetQuery*Object*(), notably the result with QUERY_RESULT.

There is a query result buffer (buffer target GL_QUERY_BUFFER) to place query results in server memory.

Timing
  • gl[Begin|End]Query(GL_TIME_ELAPSED): time elapsed query
  • glQueryCounter(): timer query

Both record time after all previous commands have been fully realized (Note: there is an error in the API reference of glQueryCounter()).

Note: The synchronous command glGetInteger*(TIMESTAMP) returns the GL time after all previous commands have reached the GL server, but have not yet necessarily completed (by opposition to a timer query, providing time after completion). This distinction allows to measure latency.

Buffer Objects

Buffer objects contain a data store holding a fixed-size allocation of server memory, and provide a mechanism to allocate, initialize, destroy, read from, and write to such memory.

The data store of a buffer object is created via:

  • gl*BufferStorage() to obtain an immutable-size buffer (content is mutable).
  • gl*BufferData() for a mutable buffer.

Buffer data can also be:

  • modified
  • mapped (for access in client address space),
  • invalidated (glInvalidateBuffer*Data(), data then have undefined value)
  • copied
  • queried (glGet*BufferParameter*() for parameters, glGet*BufferPointerv() for mapped pointer, glGet*Buffer*Data() for data)

Buffer usage is dictated by the target to which it is currently bound.

Buffer objects created by binding a name returned by glGenBuffers to any of the valid targets are formally equivalent.

(i.e. a buffer can be bound to distinct target at different points).

  • Array buffer is a buffer from which generic vertex attributes are fetched. (i.e. the buffer target is GL_ARRAY_BUFFER)
  • Element buffer (or Index buffer) is the buffer from which vertex indices are fetched (target is GL_ELEMENT_ARRAY_BUFFER).
  • Uniform Buffer Objects are buffer objects used by an application to store uniform data for a shader program. The shader program access the storage via GLSL uniform blocks (the GLSL grouping of uniforms) (target is GL_UNIFORM_BUFFER). The link is made via a uniform buffer binding location.
  • Shader Storage Buffer Objects are accessed via GLSL shader storage block (target is GL_SHADER_STORAGE_BUFFER).

Indexed targets

Some GL buffer targets are indexed. For indexed targets, there is an array of buffer object binding points, as well as the usual single general binding point (like for non-indexed targets).

A buffer can be bound to the binding point at binding index with glBindBufferBase(target, index, buffer) (or glBindBufferRange()). Binding index is the index into an indexed buffer object target. Not to be confused with the index of the resources in GLSL programs (e.g. block index for an interface block)

Indexed targets:

  • GL_ATOMIC_COUNTER_BUFFER
  • GL_TRANSFORM_FEEDBACK_BUFFER
  • GL_UNIFORM_BUFFER
  • GL_SHADER_STORAGE_BUFFER

Notes: Binding to an indexed binding point is required for specific use by GL (notably, to access the buffer in a shader program), and also binds to the general binding point (thus unbinding previously bound buffer). Non-indexed binding (glBindBuffer()) can also be used for indexed targets, it only binds to the general binding point.

In both case, the general binding point allows to use general buffer object manipulation functions, whereas indexed binding usually means that the buffer content will be used (rather than just modified).

Vertex Array Object

Vertex Array Object (or VAO) capture state for rendering. A VAO represent a collection of sets of vertex attributes, each set values (the actual data) are stored outside the VAO, as an array in a buffer object data store (TODO is the buffer called vertex attribute array?).

Binding the VAO restore all captured state at once:

  • Vertex format
    • enabled arrays (glEnableVertex[AttribArray|ArrayAttrib]()).
    • data type, element count, normalization
    • relative offset (added to the binding point base offset, thus allowing interleaving attributes)
    • associated buffer binding point (binding index <-> vertex attribute index)
  • Buffer binding point state, shared by all vertex attributes pulled from this binding point
    • source buffer object (buffer object <-> binding index)
    • base offset into the buffer
    • byte stride
    • Instance divisor, for instanced rendering
  • Index Buffer binding (The last buffer object bound to GL_ELEMENT_ARRAY_BUFFER while the VAO was bound)

Important: if the array corresponding to an attribute (required by a Vertex Shader) is not enabled, the corresponding element is taken from the current attribute state. (glVertexAttrib*() family of functions allow to set the default state for an attribute, which is used when the attribute array is not enabled)

Shader Objects & Program Objects

Shader objects to be used by one/several of the programmable stages are linked together into a Program object. Shader programs executed by these stages are called executable, all information necessary for defining each executable is encapsulated in a program object.

Interface blocks

Interface blocks are GLSL side constructs. They group input, output, uniform (in uniform blocks), buffer variables (in shader storage blocks).

storage_qualifier[in / out / uniform / buffer] block_name
{
  <members>
} instance_name;

in, out can only be used between shaders stages (not as vertex shader input or fragment shader output). Uniform blocks and shader storage blocks are collectively called buffer-backed blocks: the storage for their content come from a Buffer Object. Those buffers are indexed.

Program Interfaces & Introspection

An executable communicate with other pipeline stages or the application through a variety of interfaces. Each interface has a list of active resources (built at compile time).

GL provides functions to query properties of the interfaces of a program object:

  • glGetProgramInterfaceiv() queries properties of the interface itself (e.g. number of active resources, maximal resource name length…)
  • glGetProgramResourceiv() queries properties of the active resource at index in programInterface.
  • glGetProgramResourceName() queries the name of active resource at index in programInterface.

For interface blocks, the active resources are the blocks themselves, not the variables they group:

  • GL_UNIFORM_BLOCK active variables are accessible via the GL_UNIFORM program interface.
  • GL_SHADER_STORAGE_BLOCK active variables are accessible via the GL_BUFFER_VARIABLE program interface. (consistent with the name buffer variable for individual variable in a shader storage blocks.)

Program Pipeline Objects

A program pipeline object composes a pipeline by using shader stages from several program objects, which must have the GL_PROGRAM_SEPARABLE parameter set. (glProgramParameteri()). Note: they build on top of program objects, since they take shader stages from them. Program pipeline objects allow to combine stages without requiring one program object instance for each combination.

Textures

  • Pack: read from OpenGL
  • Unpack: write to OpenGL

glTexStorage*() specifies the internal format, i.e. the format OpenGL will use to store the texture data. Note: For the internalformat parameter, use sized formats to be explicit (instead of letting implementation decide).

glTex*Image*() is used to load data into an allocated texture. The format and type are describing the data provided by the application.

Textures have a target. Unlike buffers, a texture must always be bound to the same target it was initially created with.

Using textures with GLSL samplers

Texture names (the GLuint returned by glGenTextures()) have to be bound to texture image units (sometimes just called texture unit, warning: image unit are distinct, corresponding to image uniforms below), Binding a texture with glBindTexure() also binds it to the currently active texture image unit set with glActiveTexture(). GLSL shader programs can sample the texure via sampler uniform variables (whose type must match the texture target), set to the texture image unit with glUniform(). Warning: A sampler uniform variable is a GLSL type, distinct from a Sampler Object, which is an application object encapsulating sampling parameters.

Image load/store

image variables are GLSL uniform variables allowing arbitrary read/write to/from image within a Texture. A given level (“image”) in a texture is bound to an image unit (distinct from texture image unit) with glBindImageTexture(). The uniform value of an image GLSL variable (of matching type) is set to the same image unit.

Framebuffers

A framebuffer is a collection of logical buffers (color, depth, stencil) used as destination of rendering operations, or source of readback operations.

Framebuffer Objects (FBOs) are user-created. Their logical buffers reference individual framebuffer–attachable images, either from textures (render to texture) or from renderbuffers (off-screen rendering), attached to a set of attachment points. An FBO must be complete to be used for rendering.

The default framebuffer is provided by the system upon context creation. It usually consists of multiple colors buffers ([front|back][left|right] buffer), depth buffer and stencil buffer.

GL has 2 active framebuffers:

  • draw framebuffer (destination for rendering operations)
  • read framebuffer (source for readback operations).

The separation between Draw and Read framebuffers exists so the Read frambebuffer can be blitted to the Draw framebuffer.

Buffers

  • Color buffers can have up to 4 color components (RGBA) per pixel. All framebuffers might have multiple colors images that could be read/written.
  • Depth buffer pixels are a single value (unsigned int or floating point)
  • Stencil buffer pixel are single int value.

Renderbuffer

Renderbuffer objects contain images, intended to be used with Framebuffer Objects. Unlike textures, they are optimized for use as render targets (i.e. when there is no need to sample the image), and natively accomodate multisampling.

Vertex specification

glspec 10 p337

The process of specifying attributes of a vertex and passing them to a shader is referred to as transferring a vertex to the GL.

Vertices are specified with on or more generic vertex attributes (1, 2, 3 or 4 scalar values). Vertex shaders access an array of 4-components generic vertex attributes, this array size is MAX_VERTEX_ATTRIBS. They are loaded into the shader attribute variable bound to the generic attribute index ([0..MAX_VERTEX_ATTRIBS[).

glspec 10.2.1 p348

Matrices are loaded into these slots in column major order. Matrix columns are loaded in increasing slot numbers.

Current attribute values are transferred for a vertex when there is not vertex array enabled for a required attribute. The value can be changed by glVertexAttrib*(), and queried with glGetVertexAttrib*().

Vertex Arrays

Vertex data can be placed into arrays (Buffers with target GL_ARRAY_BUFFER). All the state to represent the vertex arrays is stored in a Vertex Array Object: it encapsulates all state related to the definition of data used by the vertex processor, and notably collects the buffer objects to be used by the vertex stage.

glEnableVertexAttribArray() (DSA variant: glEnableVertexArrayAttrib()) enables an individual generic vertex attribute array in the selected VAO.

Separate format and data source

Specifying the organization of data store of generic vertex attribute for a VAO:
glVertexAttrib*Format(attribindex, size, type, normalized, relativeoffset) (DSA variant: glVertexArrayAttrib*Format()).
This stores in the VAO that the generic vertex attribute at index attribindex will be fetched from an array of described format, starting at relativeoffset (allow interleaving different attributes in a single buffer).

Associating a vertex buffer object to a VAO’s buffer binding index:
glBindVertexBuffer(bindingindex, buffer, offset, stride) (DSA variant: glVertexArrayVertexBuffer()).
This provides a stride and offset for a buffer and attach it to the specified binding index of selected VAO.

Associating a vertex attribute to a VAO’s buffer binding index:
glVertexAttribBinding(attribindex, bindingindex) (DSA variant: glVertexArrayAttribBinding()).
The vertex attribute at attribindex will be fetched from the buffer currently attached to bindingindex of the selected VAO.

Vertex Attribute Divisor

A generic attribute is instanced when it divisor is non-zero. In this situation, the attribute advances once per divisor number of instances (implies instanced rendering). By opposition, attributes with the default divisor 0 advance once per vertex.

Divisor is set with glVertex*BindingDivisor().

Rendering commands

Rendering commands perform rendering into a framebuffer. They are:

Drawing commands

see: glspec 10.4

Drawing commands, of the form gl*Draw*(), send vertices through OpenGL to be rendered.

Some parameters are shared by most commands:

  • GLenum parameter mode is the primitive type.
  • GLsizei parameter count is the number of vertices that will be tranferred.

gl[?Multi]Draw[Arrays|Elements][?Indirect]()
gl[?Multi]DrawElementsBaseVertex()
glMultiDraw[Arrays|Elements]IndirectCount()

  • Arrays|Elements:
    • Array: fetches in contiguous sequence from enabled vertex arrays.
    • Elements: fetches from enabled vertex arrays at indices provided by the buffer bound to GL_ELEMENT_ARRAY_BUFFER. Parameter type indicates the integral type of the value in the element array buffer.
    • In both cases, the index of the current array element may be read by a VS via gl_VertexID.
  • Multi: is equivalent to issuing parameter drawcount draw commands at once, parameters becoming pointers into arrays of drawcount elements. VS can read the index of the draw via gl_DrawID.
  • BaseVertex: the value of GLint parameter basevertex is added to the indices read from the element array buffer before pulling the vertex data. It allows to load vertices for distinct models in the same GL_ARRAY_BUFFER without requiring to re-index subsequent models. (Note: this is only available for Element variant, because Array already takes a first parameter).
  • Count: drawcount is now a byte offset into the (server space) buffer bound to GL_PARAMETER_BUFFER, where a single GLsizei value is read to provide the actual draw count.
  • Indirect: read some of the draw parameters from server space memory, from the buffer bound to GL_DRAW_INDIRECT_BUFFER, offset by parameter indirect bytes.

Instanced

gl[DrawArraysInstanced[?BaseInstance]()
glDrawElementsInstanced[?BaseVertex]}[?BaseInstance]()

  • Instanced: the sequence of vertices will be drawn parameter instance times. Instanced vertex attribute array (non-zero divisor) will advance by one every divisor instances. VS may read the current instance index via gl_InstanceID.
  • BaseInstance: parameter baseinstance specifies the first element within instanced vertex attributes (i.e. an index offset).

Indirect

GL_DRAW_INDIRECT_BUFFER should contain data matching the corresponding struct (as tightly packed 32-bit values):

  • gl*DrawArraysIndirect*(): 
    struct DrawArraysIndirectCommand
{
    uint count;
    uint instanceCount;
    uint first;
    uint baseInstance;
};
  • gl*DrawElementsIndirect*(): 
    struct DrawElementsIndirectCommand
{
    uint count;
    uint instanceCount;
    uint firstIndex;
    int baseVertex;
    uint baseInstance;
}

In case of glMultiDraw[Arrays|Elements]Indirect(), the buffer, offset by indirect bytes, should contain at least drawcount instances of the struct. Each instance being stride bytes appart (0 meaning tightly packed).

Range

glDrawRangeElements[?BaseVertex]()

Note: not usefull anymore, because the vertex data and indices live in server memory.

  • Range: all element (indices) read from the element array buffer must have a value between start and end parameters (prior to adding basevertex).

Compute

TODO

glDispatchCompute[?Indirect]()

  • Indirect: read parameters from buffer bound to GL_DISPATCH_INDIRECT_BUFFER.

Extensions

  • ARB: Architecture Review Board (those are still considered extensions, not core)
  • KHR: Khronos (I think those are core)

Glossary

  • DSA: Direct State Access. The ability to modify the GL object states without binding them. Introduced in OpenGL 4.5
  • EGL: Mobile and Embedded Device Bindings
  • FS: Fragment Shader
  • GLX: X Window system Bindings
  • MRT: Multiple Render Targets
  • Render Target: Although not defined by the GL spec, render target is the collective term for either a render buffer or a texture being used as the target for rendering (see: https://stackoverflow.com/a/27186611/1027706)
  • TCS: Tessellation Control Shader
  • TES: Tessellation Evaluation Shader
  • VAB: Vertex Attribute Binding (ARB_vertex_attrib_binding), the ability to change the mapping between vertex attribute and vertex buffer binding, and to specify vertex attribute format separately.
  • Varyings: older name for the shader input and output variables between the vertex and fragment stages.
  • Vertex Attribute: user-defined inputs for vertex shaders. They are passed via Vertex Arrays from data stored in Buffer Objects.
  • VAO: Vertex Array Object is a list of the buffer objects used by the vertex stage, and how they map to Vertex Attributes in the Vertex Shader. Warning: “Vertex arrays”?? or “Vertex attribute arrays” seem to refer to the buffer storage for the vertex data, not the VAO!
  • VBUM: Vertex Buffer Unified Memory (NV_vertex_buffer_unified_memory), a generalization of bindless texture to buffers.
  • VS: Vertex Shader
  • WGL: Microsoft Windows Bindings