You'll get a better flavor if you use a permanent filter. However, you'll also get some fine silt/sediments. This is not really a problem...but you wouldn't want to drink the coffee to the last drop though. The sediments tastes really nasty. Also, if you will be going with a permanent filter, get the metal one. The plastic one somehow imparts a slight aftertaste.
If you use a paper filter, you will not get those nasty sediments. However, paper filters trap the essential oils that are released during the brewing process. This mutes the flavor a little bit.