Alternatively, should both versions of a tag be created and made synonyms ?
On Stack Overflow, I think the singular dominates but there's no agreement. Most newer Stack Exchange sites tend to favor plural tags for countable objects, e.g. compilers, debuggers, reserving the singular for concepts (compilation, debugging). I propose to follow this convention: use the plural for countable nouns. That is what was adopted with near-consensus on English Stack Exchange. It goes along the intuition that a tag name is what you use to complete the sentence “this question is about ____”.
Generally there's no need to create both versions, and indeed the engine won't let you. As soon as one version of the tag exists, you can't create the other one (e.g. now that debuggers exists, if you try to create debugger, you'll get an error message).