For any lover of Antiquity, the Louvre will in Paris be one of the main places to visit, but there is more to see. Along the Rue Monge, you will find the remains of an ancient amphitheater. Where gladiators once fought, you will now find a nice park where people are playing a more innocent sport, pétanque.
If you walk to the west along the Rue des Écoles (as it happens, an ancient road), you will reach the Musée de Cluny, the medieval museum of Paris. One of its largest rooms, however, used to be the frigidarium of one of the three known bathhouses of ancient Lutetia. You will see some ancient sculpture, including Paris’ most famous monument from the Roman age, the Pilier des Nautes. There’s also a well-known statue that was once believed to represent Julian the Apostate, but is in fact a priest of Serapis from the age of Hadrian. The room itself, by the way, was once identified as the place where Julian was proclaimed emperor.
To be honest, the Roman remains of ancient Paris are not extremely impressive and should not be your only reason to visit the Musée de Cluny. It’s a medieval museum, after all, and the real attractions are the stained glass of the Sainte-Chapelle, ivories, tapestries, and sculpture.