Tutu's grandson, Xabiso Gxashe, spoke at the service, thanking his grandfather and saying he loved and missed him.
The Mayor of Johannesburg, Mpho Phalatse, was seen laying a wreath during a ceremony.
The anti-apartheid hero Tutu died aged 90 on Sunday (December 26).
His funeral will be held on Saturday (January 1) at Cape Town's St George's Cathedral, from where he led marches and campaigns to end apartheid.
He will be cremated after Saturday's funeral service and his ashes interred beneath the floor of St George's, the cathedral's dean, Father Michael Weeder, told Reuters.