Admittedly, this is a topic on which I am a bit out of date. My "baby" is 21. But some things don't change. Outfitting for a new baby can be very expensive, but it doesn't have to be. But whatever you do, keep safety and health in mind.
First, ask around for second hand items, particularly furniture. Do be careful of safety, particularly when it comes to 30 year old cribs stashed in someone's attic. But many people have one or two children, then get rid of their stuff, and a lot of that stuff has years of life still in it. You can save a lot on a crib by buying it second hand, however, you may want to buy a new mattress for health reasons. If not, I'd clean it thoroughly and I'd recommend being extra careful of mattress covers and pads. Many car seats can be wiped down very well, and the cloth covers washed. The same with many baby seats and carriers.
A stroller isn't absolutely necessary, especially at first, unless you plan to do a lot of walking with the baby. A high chair isn't an urgent initial item, and you can probably get by without one at all. My younger daughter went from eating in the stroller to eating sitting in my lap to sitting in a small child's chair at a child's table with her sister. The high chair, which we already had, was mostly used for her to play in next to us when we washed dishes or baked bread so she could join in. A good baby swing can be a wonderful thing, but isn't necessary. Keep in mind many of these things may only be used for a few months when you consider the investment. A baby pack, front or back, or a sling can be great for a small baby, but most of these are washable which make them really good candidates for second hand purchase. I loved a bouncer seat over a cradle or a rocking car seat, and I think they're cheaper too, if you don't get fancy.