Magic weapons - the stuff of legend, or the stuff of min/max munchkinism? It all depends on how you use it and portray them in the game.
There are two basic factors I can think of that would influence a player to choose for his character to use a particular weapon:
1) Effectiveness in combat. A 1d8 weapon is better than a 1d6; a +2 is better than a +1.
2) Because, in a role playing aesthetic sense, it's cool. It complements or expands his or her conception of the character, adds flavor and flair, and connects the character to the campaign world.
The first is built right into the D&D game, and in the absence of deliberate emphasis on the second, is probably going to be the criterion to which most players default. If that's how you enjoy playing the game, nothing else need be done. The truth of the matter is that combat stats are always going to factor into weapon choice. If you want them to be less of a factor, you just need to introduce some other factors that a player will have to weigh against the power of the attack bonus.
Two primary non-game mechanic ways to add color to weapons come to mind: physical description and backstory.
What does the weapon look like? Is it plain or ornate? Is it made of typical or exotic materials? Is it engraved or inlaid with designs or runes? Is it adorned with precious stones, feathers, shark's teeth? Is there a motif or theme to it? A google image search returns pages and pages of fantasy swords, maces, axes, daggers, or whatever other weapon you care to see, if you're in need of inspiration. If your players are fond of visual aides, print them off and paste them to 3x5 cards to hand out when the party finds the weapon. (If you're of a Lawful persuasion, you may want to stick to images labeled for re-use, but hey, I'm not here to judge.)
Who made it? For whom was it made? Was it made for a specific purpose? Where and how long ago? Who has owned it and what has happened to it since then? What notable deeds have been accomplished with it? Does it have a name?
Ideally, both of those facets should be linked together, and possibly linked with the basic weapon stats and powers as well. You can use any one of them as a starting point.
A few examples:
A sword+1 of charming commissioned by a dashing, romantic swashbuckler. A slender elegant weapon with engraved thorny vines wrapping around the hilt and running the length of the blade. In the pommel is a large ruby cut to resemble a rose blossom.
A mace+1, +3 vs. magic users, crafted by a church hierarch for a famous witch hunter. The haft is of plain, polished white oak with a sacred verse condemning the blasphemies of arcane magic carved along its length; the head is an elongated ball of cold-wrought iron plated in silver and engraved with runes of power.
A dagger+2 of holding commissioned by a master thief to assist her in daring robberies. The thin, straight blade is made of dull, non-reflective steel. The hilt is wrapped in fine wire in the pattern of a spider's web, with a round onyx set in the pommel.
A battle axe +2 made for a cruel warlord. A wickedly sharp crescent shaped blade of gleaming steel with ornate inlays of black metal depicting demonic fanged faces, topped with a long spike. The haft is of blackened ironwood.
It's not difficult to imagine players choosing or eschewing each of these weapons on the basis of its appearance and whether or not it harmonizes with the player's concept of his character. A brooding barbarian might despise the rose sword, while a noble knight might be reluctant to take up the warlord's axe.
It's even conceivable that players might become attached to a weapon that's become part of a character's signature style, and won't want to part with it even for a statistically superior one.
But wait, there's more! What if you never told players exactly what the combat bonuses of their weapons were? Just keep a record in your own notes, and apply them as needed. Surely that would tilt the balance of coolness vs. combat stats even further in the direction of the former.
Of course, you need to tell your player something about what effect his sword is having, which brings up another facet of description. Unfortunately, as far as I know, no rule book has ever really explained how magical arms and armor work in-world. They give bonuses to hit and damage or to armor class, but those pluses are completely disassociated from whatever in-game effects actually produce them. You'll need to have an in-world explanation of how they work, so you'll have something to describe to players without simply reciting numbers. Even if keeping weapon stats secret from your players isn't your style, I think being able to articulate what's actually happening when Cedric the fighter swings his enchanted sword at the dragon adds to the experience. Here are a few ideas that come to mind.
1) The weapon augments and synergizes with the combat skill of the wielder. Perhaps over years of being used by heroes of legendary skill, their combat techniques imprint upon the weapon. A new user finds himself moving more smoothly in combat, and reflexively using maneuvers he hadn't previously mastered, as if the weapon itself were guiding his hand. (This way has some awesome potential in what it implies about really powerful weapons. A sword+5 may very well have been used by not one, but several of the most renowned heroes or villains in the campaign world's history through the ages. Perhaps a non-magical weapon could even spontaneously become enchanted in this way.)
2) The weapon or armor may be unnaturally hard and durable, able to hold a keener edge or point (blades or piercing weapons,) pound through defenses (blunt weapons,) or resist those things (armor) better than those made of ordinary materials.
3) The weapon might discharge pure magical energy, similar to that of a magic missile, into the target on contact, breaking through defenses and causing more harm.
4) The weapon may be fortified with some substance that's anathema to a specific creature or class of creatures, like holy water for undead or wolfsbane for lycanthropes.
Of course, there doesn't have to be one way that all magic weapons in the campaign work. Some might be 1's, some 2's, some 3's, or something else entirely. The exact method is part of the flavor of a particular weapon.
And that's it for magical armaments. I didn't see much reason to rehash scarcity here, and I think I'll save the topic of applying conditions and costs to weapons for another post.