Yes, Lord Shiva knew that Mohini was Lord Vishnu only, but due to Bhagavan’s maya, He temporarily forgot that Mohini was Bhagavan. After the Samudra Manthana, Lord Shiva approached Lord Vishnu. Because when Lord Shiva had drunk the Halahala poison, His throat was hurting, so He did the japa of the Achyuta, Ananta, and Govinda mantras and then finally got rid of that pain.
So, Lord Shiva wasn’t able to see the Mohini roopa of Bhagavan. Therefore, He approached Bhagavan in Vaikuntha, along with Umadevi, and asked Him to show His Mohini roopa. Bhagavan said, “O Shiva! My Mohini roopa was for deluding the demons and not for anything else. I recommend that you not see that roopa, because it is a very bewildering form of mine.”
But then, Lord Shiva still insisted upon it. Lord Vishnu finally agreed and said, “Wait a minute. I would come.” And went away. Suddenly, the environment changed. There was a beautiful garden. Lord Shiva saw a beautiful girl playing along with a ball, tapping and bouncing the ball and catching it. Her laughter was itself too much bewildering that it may capture anyone.
Her anklets in the Her feet made a ‘Ghiluk’ sound. She wore neat and bright clothes, and was decorated with all sorts of ornaments. She had hair that was rich black and had a hairbun which was decorated with different flowers. Some curls of Her hair spread over Her forehead. She had a braid that was decorated beautifully. Her belly was slender, making Her even more enchanting.
Her eyes were as long as the petals of a lotus. She wore kumkuma tilaka on Her forehead, and Her lips were red like bimba fruit, and Her teeth shone like pearls. She was the color of rain-filled clouds. She had bangles in Her hands, and Her skin was so soft that even if you took a lotus petal and touched it on Her skin, it would create a scratch.
But one must note that Bhagavan is softer than the softest and harder than the hardest. Such an enchanting form was the Mohini form of Lord Vishnu. As Her name itself suggests, it was a form taken for bewildering (Mohana) the demons. Lord Shiva saw this form of Lord Vishnu and ran towards Her. He ran behind Mohini and grabbed Her by Her braid and hugged Her. But then, Mohini turned into Bhagavan Vishnu.
Lord Shiva realized that this girl was none other than the Mohini roopa of Bhagavan Vishnu. Means, that form is so bewildering that, it even bewildered Mahadeva, who is known to be as Vairagya Shikhamani, and has a very peak level Vairagya. Lord Shiva is Vairagya Shikhamani, and there is no doubt about this, but this was to show how powerful Bhagavan’s maya is.
As Krishna Himself says in the Bhagavad Gita:
दैवी ह्येषा गुणमयी मम माया दुरत्यया ।
मामेव ये प्रपद्यन्ते मायामेतां तरन्ति ते ॥ 14॥
My divine energy Maya, consisting of the three modes of nature, is very difficult to overcome. But those who surrender unto Me cross over it easily.
[Bhagavad Gita, Chapter 7, Verse 14]
Lord Shiva, being the greatest devotee of the Supreme Bhagavan Vishnu, was free from all kinds of attachments and materialistic things, but to show Him Bhagavan’s Mohini roopa, and to show how powerful Bhagavan’s maaya was, all this happened.
Although Lord Shiva knew that Bhagavan Vishnu had assumed the Mohini form, upon seeing that extraordinarily enchanting form, He became bewildered by Bhagavan's Maya. This demonstrates the inconceivable power of Bhagavan Vishnu's Maya, not any deficiency in Mahadeva's greatness or vairagya.
But one must note that Lord Shiva is infinitely superior to us and is the greatest devotee of Bhagavan Vishnu, thus known by the name Vaishnavottama or Bhaagavatottama. He is the one who gives the Shaastra paatha to the devatas along with Umadevi.
He is the niyamaka for the bodies of the devatas like Indra, Chandra, etc. and is the head for them. He is the controller of the mind, and because of His blessings only can we gain control over our minds. He, along with Umadevi, handles the department of the mind. So, Lord Shiva is definitely and infinitely higher than us, and we are nothing in front of Him.
We cannot control Mahadeva by mohana or sammohana, because He is beyond that, and can only be pleased by pure devotion. Without Bhakti, Lord Shiva won’t even look at you. Umadevi too attained Lord Shiva through devotion only. But this was the Supreme Bhagavan’s maaya, in front of which nobody can survive, neither Lord Brahma, nor Lord Shiva.
Hope this helps!
Jay SriRam!
ShriKrishnarpanamastu!