Even if you're improvising the fact that beforehand you know certain things will work helps you make those improvisations successful. It really helps to have a certain amount of knowledge about musical structure.
I know it may seem surprising to people but learning dialog that has a conversational flow to it is not that difficult.