Edit: It seems my initial question (Why are there no insulating heatsinks?) was based on a false premise, and there are in fact insulating heatsinks--I just wasn't able to find them with a cursory search. So instead, I'm changing this to ask about their rarity instead.
Heatsinks seem to be almost universally made of aluminum, copper, or some combination thereof. This makes sense; aluminum and copper are easy to work and have high thermal conductivity. But diamond has one of the highest thermal conductivities of any known substance--it's obvious, of course, that diamond of the type suitable for use as a heatsink would be inordinately expensive to say the least, as it would probably have to be a single gem-quality crystal, but would it not be possible to use, for example, cubic boron nitride, which has a similar thermal conductivity?
And yes, the manufacturing difficulties with making a large single crystal of c-BN would probably be about the same as making a large single crystal of diamond, but I expect the end price wouldn't be as much because there's no De Beers group to come after you for boron nitride. And there are surely other nonmetallic compounds that have good heat conductivity, and some of them would presumably be better suited to manufacture. I doubt they'd be able to even approach the price point of extruded aluminum, but sometimes you do need higher performance.
So, in summary, my question is: Is it only cost that makes nonmetallic heatsinks so rare, or are there some other drawbacks that make them less desirable outside of the most esoteric of applications?