Cleaning removes dirt. It does not repair a belt that has been damaged.
Look along the whole belt while it is pulled out. Fraying or cuts at the edge, a stiff or hardened section, a glazed or melted patch, or webbing that has gone thin are all reasons to stop cleaning and have it looked at. So is a belt that retracts slowly, sticks, or lets go of its tension in the middle of its travel, or a buckle that no longer latches with a clear click and a firm hold.
Vehicle makers generally also say a belt that has been through a significant crash should be replaced even if it looks fine. Your owner's manual spells out its own rule. A dealer or a mechanic can inspect and replace belts. That is not a detailing job, and we would not suggest otherwise.
If you are not sure whether a belt is worn or just dirty, clean it first as above. Dirt hides damage, and a clean belt shows it.