complete 6" ducting?

[charleydoodle]

  I just bought a HF 2HP unit.  I am using 6" SDR PVC duct.  My question is, does the 5" running from the blower to the filter/bag assembly affect the cfm enough to upsize it too.  Since I'm at the beginning, I would rather do everything possible to ensure the best possible performance.  I also have all my tools on one wall initially due to the layout of my shop.  I've realized in this process however that this should make dust collection simpler.  I have my RAS, SMS, DP, BS, and BS all within 18' on the north wall with the TS and Jointer protruding perpendiclarly 8'.  Is it worth cutting out the fittings to upsize them to 6" on the blower to filter/bag junction?