Because this is talking about an event in the past that did not happen, you have to use the past-perfect tense for the "if" part of the sentence, and the "would have + past participle" for the second part of the sentence. It's confusing, but if it makes you feel any better, most native English speakers don't worry about this, and it might even vanish from the language in a few generations.
@joefumai Thanks a lot! I ignored that it didn’t happen! Thought you’re right about grammar might vanish in future, I still have to learn the correct one. It’s an important language culture:) (But I still hope I’ll have colloquially chat with foreigner in someday😢)