Archaic expression that still lives.
I remember my old Norwegian aunt expressing surprise or confirmation many years ago by declaring "by gum". I haven't heard that expression since then, until today…from the mouth of the Director of Marketing at the historic Capital Hotel in Little Rock, Arkansas.
My curiosity got the best of me, so I did a Google search to discover its source and usage. Here's what I learned:
It's an exclamation of surprise, a euphemism for "By God" or "By golly". It has been used since the early 19th century; and is still used in the north of England, but is considered archaic elsewhere.