The term is directly related to the terms Aye and Aye Aye
It was a phrase that the people of the north of the Britians, developed as a sort of war cry when Vikings were attacking their coastlines.
The Britians used to yell the words "Yi-Ei-Man" to the attacking Vikings, which mean literally "No, Always, Man"
The phrase was intending to imply that the Britains would never join the Vikings crews and would fight to the death down to the last man.
In modern times the words have completely lost all meaning and the phrase is just a sort of slang way of agreeing to something - ironically exactly the opposite of what it should be.