Speaker doesn’t expect long-term economic peril with Brexit move