Where to build down-payment for next house?

I see my family either building a house within the next calendar year. This got me thinking about where the down payment funds are coming from, as most of our money is in retirement accounts & brokerage accounts. Here's a quick breakdown of estimated numbers.

PNC Checking - 10k

PNC Savings - 10k

Vanguard HYSA - 50k

Vanguard Brokerage - 200k

Vanguard Retirement Accounts - 150k

Raymond James Brokerage - $150k

Most of the funds in the brokerage accounts are in S&P500 funds, specifically VTSAX, etc. Raymond James is more diversified as it's from a former financial adviser I had.

I have a $250 auto-investments going into VTSAX in Vanguard every week. Should I pause this for the time being and put it into HYSA?

Now for a 20% down payment on a $600-700k house, I'm wondering where this would come from. For the next year, should I stop investing in brokerage and just put more into the HYSA? If I sold the funds I'd have a hefty tax bill next year.

Also, we're not definitely selling our current property. We're at a 2% interest rate and may keep it long term to aid in retirement.

What's the best course of action here?