Some museums appeal to you because they are interesting, like Delphi. With some exceptions, its sculpture is not extremely remarkable, but the museum tells a good and important story about ancient religion. Something similar can be said about the museum of Dion: a thought-provoking cross-section of the type of art one might have seen in a small town in a Roman province. The collections of other museums appeal to your sense of beauty, like the Acropolis Museum. Or the Byzantine Museum in Athens.
You don’t have to be a Christian to appreciate that the icons and reliefs are fine-looking. There’s a splendid mosaic of the Mother of God, tenderly looking at her Child. There are gorgeous mosaics and sculptures from the Athenian churches of Late Antiquity, like the Parthenon. You will see nice coins and impressive manuscripts – a purple codex, a list of privileges from Monemvasia on the other end of a room, and a splendid gospel of John somewhere else. I loved the priestly garments from Egypt. Byzantine art induces some kind of serenity.
About interesting things, you can write something. To describe beauty, however, words are inadequate. It has always intrigued me that art historians start to explain art, catching a presumed meaning in language. But art is sublime and transcends spoken words. And whereof one cannot speak, thereof one must be silent. Go to the Byzantine Museum, experience the works of art, and don’t try to explain what it means to you.