How to insulate this rim joist area?

There appears to be some blocking and/or floor joist blocking the rim joist. It some areas it’s brick that is flush with the floor joist. The bricks are cooler to the touch compared to the wood. Is this in fact my rim joist? If not, how should I go about insulating? Just spray over the area or do I actually need to do any more demo. I have zoomed out and zoomed in photos. The house is an early 1900’s Tudor w/ exterior stucco and plaster walls. Highly doubt there’s any insulation on the interior walls and could be ballon framing for all I know.