You can remove all that if you wish. The purpose is to mix air from the exhaust and other air to a temprature of 140 degrees. Your carberator is jetted to mix air fuel correctly at this temprature for best milage. The vacuum pot is to change this to coldest air possible when higher preformance is required. It also eliminated any carb ice problems which can possibly be encountered with unmixed air.
When upgrading an air intake for performance, you can't beat outside air.