You forgot about the butterfly. Even if it looks like nothing could change, something could due to a bit flipping in memory, which would allow the program to halt.
Right. Any statement that the program is in an infinite loop will usually be subject to error.
Of course, the real question anyone asked to develop such a program would be what runtime is equivalent to an infinite loop. Any real-world program must have a runtime bound that is predictable on invocation or it's useless.