After the supplement was printed, Méchain published a letter of correction saying that M102 is “nothing but an error. This nebula is the same as the preceding No. 101. In the list of my nebulous stars communicated to him M. Messier was confused due to an error in the sky-chart.”
What’s more, two positions of M101 do exist. The 1781 position of M101 places it 13/4° to the west of its modern position. It also places M101 a mere 40 arcminutes from the 6th-magnitude star 86 Ursae Majoris. This, of course, would explain why Messier, in his description of what he thought was M102, writes, “Close to it is a sixth-magnitude star.”
On the other hand, the 1781 position of M101, when converted to modern coordinates, differs from the one in the NASA Extragalactic Database by a mere one arcminute. This could be the position of M101 published in Messier’s catalog to which Méchain referred.
Despite Méchain’s letter and that the above research provides a viable solution to the mystery, others believe that Méchain’s observation of M102 best describes spiral galaxy NGC 5866.
On the SEDS website (http://messier.seds.org/m/m102d.html), Hartmut Frommert presents an equally intriguing argument for NGC 5866 to be M102. The full article and a supplement are well worth reading, and I encourage you to do so.