So... how did this work?
I've been comparing the Apple II with OSI stuff and I'm up to the color systems. The Apple II color was an ugly hack, effectively splattering a square wave into the NTSC color subcarrier almost accidentally generating color. Another hack adjusted the phase of the color burst so this square wave could optionally make the next line be the opposite color of the previous line. That's it, two colors, one color per line.
Looking at page 6 of the 540B schematic, it's clear OSI was trying to make a complete color system. You have true RGB signals coming in representing actual colors. There's the standard 3.579543 MHz oscillator with no apparent tricks. I'm guessing the 74123 is doing the color burst since it's triggered by the horizontal sync again with no tricks (a note says it's about 8 nanoseconds long but it should be more like 3, probably close enough).
But how are the different colors generated in the composite color? The RGB signals are going into the 74151 multiplexer, but what else is going into it? I see the 3.579543 MHz signal is going through a series of inverters generating signals CD0-6 and they're also going into the multiplexer. Wait, are they relying on the delay of the color signal going through those inverters to cause a phase shift in each CD signal so they'll hit all 360 degrees of the color cycle?
I'm so glad I don't do hardware.