This is correct.
Pet Speed is greatly influenced by owner's speed when out of combat and maybe some rules only apply when follow is enable...
Pet will gain every speed buff his owner has even if he didn't receive the effect himself, group speed, self speed, potion speed, sprint speed, pve bonus speed, literally everything so he can catch up to his master speed and allow him to follow without lagging behind !
I don't know how this is handled on live server, and losing your pet because he broke his realm speed or didn't received it at the hastener could be a frustration