Great question! Not much space is needed, as the brackets flex very minimally. At the very least however, there should be 3/4 of an inch of clearance to be safe. Your conclusion regarding unwanted movement is correct, as there would be a much bigger issue if that were to happen. To answer your original questions though, I’d make sure there is at least 3/4 of an inch for possible but minimal movement.