I didn't see anyone go into detail here, so I figured I'd explain:
The traditional reason was that a taller chimney will contain a taller column of hot air which will generate more draft, which is handy for the design of the furnace. Why will a taller column of hot air generate more draft, why does height matter and not volume? Well, because draft is determined by the difference in atmospheric pressure at the bottom of the chimney. Atmospheric pressure is the weight of the column of air above the ground - 1 cm2 from the ground to space is about 1kg - but warmer air, being less dense, will weigh less; air is 1.18 kg/m3 at STP and 0.95 kg/m3 at 100C. So, a tall column of hot air will weigh significantly less, meaning the bottom of the chimney will have substantially lower pressure, and thus cause substantial draft.
This is less common these days, with modern systems that have mechanically induced draft. Tall chimneys are still built, though - why? Well, for pollution dispersal. Pollutant limits are rarely absolute; they are typically measured per m3 of air or soil or water. If you can disperse the same pollutant over a wider area, your factory can release the same amount of pollutant and still meet regulatory standards. Higher smokestacks are very effective at dispersing pollutants over a wide area because average windspeed significantly increases with altitude. That, itself, is another fascinating effect, and is how soaring birds fly: https://en.wikipedia.org/wiki/Wind_gradient
And now a fun aside. Many years ago I designed and built my own woodstove from scratch, welded up out of scrap steel. It was a pretty good, modern design but it only worked half the time. Why? Lack of draft!
[dead]