The distinction between insulators and semiconductors is temperature-dependent; an insulator becomes a semiconductor when the thermal energy of its electrons is enough to permit a significant number of its electrons to transition from its valence band to its conduction band, continuously replacing those that fall back down. Even if the bandgap doesn't decrease with temperature, as they generally do, this means that any insulating material will eventually become a semiconductor at a high enough temperature.
There is a lot of work on "wide-bandgap high-temperature semiconductors" like gallium nitride and silicon carbide with applications up to 300°. But I think we can think bigger: what about semiconductors at 900° or 1200°? The Stefan-Boltzmann law gives us power dissipation proportional to T4; if at 100° we can dissipate 1 W/cm², then at 700° we should be able to dissipate 16 W/cm², and at 1200°, some 243 W/cm². This should permit substantially higher computational speeds, even despite other temperature-driven phenomena that will slow down computation.
What materials barely begin to conduct electricity at such temperatures? Quartz, of course, but perhaps also other refractory materials like lime, beryllia, thoria, urania, and magnesia. Other conduction phenomena also come into play at these temperatures, like zirconia's conduction by mobile oxygen ions.
Building devices that operate successfully at such temperatures will be challenging, among other things because they would probably be destroyed if ever allowed to cool to room temperature, and because the lifespans of structures even in fairly stable materials is human-scale at high temperatures. As discussed in Gardening machines, though, automated fabrication should reduce the importance of such durability considerations considerably.
Another difficulty is that you probably still benefit something to use as an insulator in your circuits, although room-temperature gallium arsenide integrated circuits seem to mostly do okay just using reverse-biased diode junctions.