Medium brush, an all purpose cleaner like Meguiars APC or Tesco's Daisy and patience. Bare in mind, the wetter the seat belts get, the more slowly they will dry. And if you force dry them, you might end up with streaks.
Always test a small, inconspicuous area first, to make sure they are colour fast to the product you end up using.