Quake BSP Format
From Quake Wiki
Quake levels are stored in a BSP file (extension: .bsp). BSP files are typically a result of processing simple text-based MAP format (extension: .map) that most of the Quake level editors support. As such BSPs may be referred to as "compiled maps", in contrast to MAPs.
Name of the file comes from binary space partitioning, a technique used by Quake to subdivide large spaces into smaller hulls that are later used to speed up rendering geometry of the level.
BSP files are most commonly generated in three steps, with a dedicated tool for each. First qbsp tool produces majority of file contents (vertices, textures, entities, models, etc.). Then vis tool computes visibility data (potentially visible sets). Finally light tool bakes lightmaps used by the renderer.
Many versions of BSP file are in use by various engines. These engines typically share common ancestry with the original Quake engine (which is true for later id technology as well as most, if not all, Valve games.) BSP version supported by the original engine is sometimes known as BSP29 due to version identifier embedded in the file (29 or 0x1d).
BSP file is composed of header with version number (four bytes: 0x1d, 0x00, 0x00, 0x00), fixed list of lump definitions and variable number of lumps. Lump represents structured data of some kind (e.g. a list of textures). Lump definitions indicate where in the file itself data of a given type is located and how much space it occupies. Number and type of lump definitions is bound to the version number. Layout is thus as follows:
[version] [lumpdef1] [... ][lumpdefN] [lumpA] [lumpB] [...]
Due to limitations of the BSP29 format, many modern Engines lifted some of the limits introducing new format. The commonly supported formats are BSP2 and its variant, Remake Quake's BSP2 format (a.k.a. 2PSB or BSP2RMQ). It is important to note however that the set of data stored in a file does not change between BSP2 and BSP2. This means that each have the same set of lump definitions but data format may be different.
Lumps affected by the BSP2 format are Edges, Faces, Leafs, Nodes, Marksurfaces and Clipnodes. Furthermore:
- 2PBS uses 32bit values instead of shorts for most except for bbox sizes (which still use shorts)
- BSP2 uses 32bits for everything and bboxes use floats
Details on BSP2 history and support across engines can be found in the BSP2 article.
With time engines added support for more data types associated with map (e.g. finer light maps, improved skyboxes, etc.). Due to a rigid nature of the file format (fixed set of lumps/lump definitions) these were shipped alongside the map. With time BSPX file format was introduced which introduces additional optional lump definitions behind the BSP29 ones (with 4 byte alignment). They differ from original lump definitions as they can be present in any order and as such require unique lump type identifier. This identifier is a unique 24 byte string (including terminating null) representing type of data being described.
Additional information on how the format is structured can be found here. Keep in mind that this is a reverse engineered Quake demo file format which may differ from the final one (demo used BSP28).