I use WM5 on an O2 XDA Exec with a 2GB card. A lot of installation files for WM5 need to run on a pc via ActiveSync (cable connection), while others need to be transferred to the PDA and ran through WM5's File Manager (like CAB files) ... so it all depends on the installation files.
As AY posted, [X] doesn't really close the application - it still runs in the background. The Task Manager is in the Programs folder, but a better way (for me) is to install Magic Button which let you close the app from the task bar. Windows Media isn't good either because of the limited media types it can play. There maybe better ones out there, but it's TCPMP I'm using as a media player.