It follows on from a similar missive made last month when non-essential shops opened for the first time.
Pubs, restaurants and hairdressers reopened last weekend on so-called "Super Saturday", while indoor gyms and other busineses will return on July 25.
In England it is currently mandatory to wear a mask while travelling on public transport, in hospitals and at GP surgeries.
However, while Scotland introduced legislation on Friday requiring the use of face coverings in shops, following a recent increase in coronavirus cases, no such rules exist south of the border.
The Government recommends that Britons take measures such as wearing a face covering if people indoors cannot keep two metres away from each other.
In the letter, the Mayor said he was "frustrated and disappointed" face coverings had not been made mandatory as London opens up.
Mr Khan also highlighted advice from the British Medical Association, which published a statement ahead of July 4 saying that it was calling for face coverings to be worn by public "as a matter of course", and statements made by the president of the Royal Society, Professor Venki Ramakrishna.
The expert has recommended that everyone should wear a face covering in public to reduce the risk of a second wave of Covid-19 infections.
The letter read: "The evidence is clear that wearing face coverings in enclosed public places can reduce the risk of spreading the coronavirus and I am disappointed and frustrated that the Government has not taken this opportunity to make face coverings mandatory in busy and enclosed public places.
"I simply don’t understand why we wouldn’t use all the tools at our disposal to try and stop the spread of this virus, so I am urging you to take more decisive action on this."
It continued: "As the President of the Royal Society notes, we are already way behind other countries in providing clear policies and guidelines on face coverings to the public. I would ask you to heed the call of our medical and scientific experts."
He argued the mandatory wearing of face masks could have a positive impact on the economy, as it would inspire confidence as a "visible signal that Londoners are willing to take the steps needed to keep each other safe".
He added: "Making face coverings mandatory in enclosed public places where you can’t keep your distance from others, like shops and hairdressers, will help us avoid a second wave, keep businesses open, and ultimately save lives."
The letter comes as Downing Street faced questions about why more Cabinet ministers are not being seen or pictured wearing face coverings.
There have been no public sightings of Prime Minister Boris Johnson wearing a face covering, while Chancellor Rishi Sunak took part in a photo op serving food in a Wagamama branch without wearing a mask.
A spokesman for the Prime Minister said a face covering was "no substitute for good social distancing practices" such as staying two metres away from others.
On the question of ministers' behaviour he said: "I don't spend my time with individual ministers but all the ministers abide by the social distancing guidance which is in place."
The Department for Health and Social Care has been contacted for comment.