To add to the previous response, the FE exhaust manifold is kept cool by being directly clamped to the cylinder head. Some "rebuild kits" provide thick exhaust manifold gaskets, but these nightmare gaskets "insulate" the exhaust manifold from the cylinder head, preventing the exhaust manifold from releasing heat energy to the coolant in the cylinder head, and causing them to warp or crack.
Thick FE exhaust gaskets are only appropriate for steel-tube headers, as steel can handle higher temperatures than cast iron. Cast iron gets soft when overheated. As always, note that many FE heads have high-exit exhaust ports, most 1966-later heads have low-exit "unibody compatible" ports, and a few 1966-later heads have neither.
The "factory original" way the FE exhaust manifold was clamped to the head (not sure about CJ/SCJ) was to either use an embossed soft steel shim gasket, or to use no gasket at all. This kept the exhaust manifold cool. The embossed steel shims are probably extinct, as I've not seen any new ones for decades. Nowadays, some folk find good luck using a very thin layer of copper-silicone applied to the face of the manifold, as the copper helps conduct heat out of the manifold and into the cooling system, while never getting hot enough to degrade the silicone. I've not tried to use silicone on an exhaust manifold yet. I find freshly surfaced exhaust manifolds clamp productively to the cylinder head.
The "hot spot" you've found at the corner cylinder might have something to do with a rusty mating surface at that location, helping insulate the manifold from the head. Fortunately, the rear cylinder is well-immersed in coolant, so I don't suspect the head is getting hot.
As an alternate theory (and not at all likely to cause your kind of single-cylinder temperature), the passenger head gasket may be on backward, preventing coolant flow behind the #4 cylinder. Fortunately, FE head gaskets have a visible "tab" which can be easily seen when peering under the hood. As long as that head-gasket "orientation tab" is properly positioned, you know the head gasket is installed correctly.