The actor tells Kelly he was later given a "pardon from the Portuguese government"