When someone switches to English
In Japan, someone who looks like they are from abroad often gets English first. にほんごは だいじょうぶです says Japanese is fine.
Many people in Japan assume that someone who looks like they are from abroad does not speak Japanese, and switch to English to be helpful — often before you have said a word. It is kindly meant, and it happens to everyone who learns Japanese. Sometimes the person switching is more nervous about their English than you are about your Japanese.
The sentence that settles it is にほんごは だいじょうぶです — “Japanese is fine”. Say it early, ideally as part of your introduction, so neither of you has to guess. If you would rather have English, えいごは だいじょうぶです works the same way.
The question you may get is the same sentence with か: にほんごは だいじょうぶですか — “is Japanese okay?”. The answer is はい、だいじょうぶです。