I have to say — I can imagine something like this happening, but I’m a litle taken aback that this concept is so well-embedded in our culture that there is a commonly-recognized term for it. This wikipedia page contains many examples of “famous” spite houses, including the little bugger pictured to the right. It will also teach you about the less-common, but still spiteful “spite fences.”
The best part about a spite house? Because it isn’t built for any practical use by its owner, there is a lot of leeway when it comes to design — you can really let your imagination soar when the structure you’re building has no intended use beyond to irritate!