How long does it take to walk again from a fractured ankle?

I fractured my ankle learning to skateboard, it was going good until it wasn’t. It was about 5 weeks ago on Wednesday. I went to the my doctor they gave me a cast. It’s been on for 3 weeks. My question is how long did it take some of you to start walking again?? I feel my feet rotating amazing, bearing decent weight on crutches. I want to get it taken off this Thursday but know it will take a lot of recovery as well. I’m scared because I need to go back to work but don’t want to rush it and injure again. I will call tomorrow but today is Labor Day so they’re not available.