It may not be unethical in the way that say, torture or stealing is, but I believe it is unethical--especially if it's something irreplaceable.
I'll admit I'm biased; I come to this topic as someone with a keen interest in history, to whom the obliteration of the past is a travesty. I think it's safe to say that what we don't know about the past is so much more than what we do know because of thousands of years of burning, looting, and warfare (to say nothing of natural decay) that's steadily erased all but the most persistent and relevant details. The ancient Greeks had a vibrant painting tradition; to my knowledge, almost none have survived. All of Aristotle's surviving works are basically lecture notes; most are lost. One can go on and on.
To me, a Fatimid serving dish or a Greek amphitheater or a Ming vase is more than just the object itself; it's the time it was made, the work that went into it, the hands it passed through, where it laid in the end. Call it mystical bullshit, but an artifact is all those things to me, something more than what it is in itself. Painting a reproduction of the Mona Lisa is easy; why don't we just burn it and make a thousand Warhol-esque facsimiles? Clearly, it's not just a painting to lots of people.