Is opening a nursery/garden center a worthwhile business venture?

My wife and I are leaving the healthcare field after 10+ years. Insurance is just ruining the ability for providers to treat people like people. We are both very passionate about gardening (specifically native gardening and ecological restoration) and have thought about opening our own native plant garden center and coaching business. From those in the field, what are the pros/cons? Can you have this as your sole source of income? We're not looking to be millionaires, just want to spend our days outside, in the garden, talking plants with people but obviously have make a living. Thanks!