There is a heart located at the top right corner of a user's profile photo. When you click on the heart, it will turn red which means you have added this user to your favourites.
By clicking this heart again, the user will be removed from your list.