Found Server takes forever

Recently i wanted to start playing again, I wanted to set up server for me and my friend. I load up the server and it stays on "found server" for like 5-10mins then finally lets me in. Its not completely stopping. just taking forever. Any way i can speed this up or fix it?

I have loaded with mods, and no mods. it doesn't matter how many or how little of mods I have . it is still 5-10 mins