Sunstone dev log 23: colour 'n' stuff
The renderer doesn’t compute a single (il)luminance value per pixel. If it did, then the image would be monochrome. We need to track luminance of different colours. A more physically-accurate renderer could compute a distribution over all (visible) wavelengths (a “spectral distribution”), but I’m going to use RGB triples instead.
In this somewhat-physically-based setting, an RGB triple represents the spectral distribution formed from the linear combination of 3 basis distributions: one for the red component, one for green, and one for blue. A colour space defines the basis. I’ll use the sRGB colour space because it’s ubiquitous in computing. In particular, GLTF assumes sRGB colour for its materials.
The sRGB basis (“primaries”) is defined in terms of the CIE XYZ colour space throug a linear isomorphism between sRGB triples and CIE XYZ triples. This transformation is for linear sRGB, after the transfer function has been applied. The primaries (in CIE XYZ) are:
CIE XYZ summarises the appearance of a spectral radiance distribution. It doesn’t summarise a spectral luminance distribution because its Y channel is the luminance of the distribution, which is calculated by applying the luminous efficiency function.
The linear map from sRGB to CIE XYZ is made of linear maps from sRGB to each of the the CIE components. Of note is the linear map to the Y channel: . Because it’s linear, scaling an sRGB value also scales the luminance channel:
This means it’s reasonable to use sRGB triples to approximate spectral distributions in the renderer. It does come at an artistic cost, however.
Because an sRGB triple is only a summary of a spectral distribution (via its isomorphism to CIE XYZ), it discards information about the actual wavelengths being transmitted, which can significantly impact the appearance of a scene. For example, consider a yellow light illuminating a red box. Let’s say this red box only reflects light in the 625–675nm range. If the yellow light is a 580nm monochromatic yellow, then the box will appear black. But if the same yellow light was produced by a mix of green and red monochromatic lights, where the red component falls within 625–675nm, then the box would appear red. sRGB can only express the latter. Here are two cool examples of this difference:
Aside (spoilers for “The Weird World in RGB”)
I rewatched the video with a friend and realised that I was surprised that the yellow GameBoy Color reflected both red and green light. I expected the yellow paint to mostly reflect light in a narrow range aroudn ~580nm.
I searched around for typical spectral distributions of yellow things, and came across What is yellow, which suggests that most yellow things reflect a fair amount of light and mostly absorb the upper end of the visible spectrum. So the GameBoy Color reflecting both red and green light is probably the rule, rather than an exception.
The article also demonstrated two “complementary spectral distributions”, where one spectral distribution was a narrow band, the other excluded this band, yet both have the same colour. That’s mindblowing.Rendering with sRGB isn’t incorrect, it’s just limited. “The Weird World in RGB” reproduces these effects in real life. So a way to think about physically-based rendering with sRGB is that an sRGB light source is a combination of 3 discrete wavelengths, and an sRGB reflectance is the proportion of these 3 wavelengths that are reflected.
Accounting for all this in my plan from last time:
For each fragment, store in the colour attachment
lis an sRGB triple, therefore it’s a spectral radiance distribution.
Compute from
lshould be average luminance, so it’s the mean of each entry’s Y channel.
Compute
For each inl, compute the normalised sensor levelThis now seems like the wrong way to do it.
I can still compute radiant exposure:
But now it looks like there are 3 sensor levels per pixel: R, G, and B. I’m not sure what the saturation point should be or how I should normalise each of them. Anyway, since I can compute RGB values I should just implement what I know so far and play around with this final part.