Every part your plug into the mobo has been designed so that there's only 1 way it can go in. You have very little to worry about with putting components in the wrong way round and such.
As far as I know, there's absolutely no link between choosing a CPU and PSU. the thing you need to consider with the PSU is if it can supply enough Watts and if it has the right connectors for your mobo and GPU.
I'd say be careful with the HDD since they're fragile. Less of a concern if you're going with an SSD.
If you're getting a 'tool free' case (recommended by Psymon

), make sure you work out how the tool free bits work before you've plugged anything in and work out how they work on one you're unlikely to use. Just to be safe

Probably goes without saying but don't use a screwdriver with a magnetic tip. Easy enough to forgot so I like to mention it.
Buy some extra thermal paste, just in case you make a mistake with installing the heat sink or it could do with some better stuff. Careful using it though, you can very easily add WAY too much.
I like to install the PSU into the case first (if it doesn't get in the way of installing the mobo), then the CPU, RAM, and GPU into the mobo before I screw the mobo into the case because it flexes more once it's on the screws which can't do it much good. Your GPU shouldn't cover any screw holes but double check.
Last thing for now, get a surge protected extension cable. Doubt I need explain that one

Best of luck with the new PC. Tell us about it's spec once you've decided
