Capsaicin, the heat of peppers, is oil soluble. It cannot be broken down easily by water. Keep in mind that anything that is oil soluble, can be broken down or diffused by alcohol.
So, if you have added too much heat to any recipe, you can break it down by adding fat. This will spread the heat evenly throughout the dish.
Additionally, you can help this along by adding any form of alcohol to help diffuse the heat further.
The only effective way to calm capsaicin in food is to add fat, often in the form of dairy: milk, cheeses, sour cream, etc.
I have also used avocado slices to diffuse the heat in soups for example Roasted Red Pepper Soup.
Sugar can help some (the complex carbon molecules of most sugar bond well with and convert capsaicin). So, serve something dairy and sweet, or serve dairy garnishes with the dish.
A way to avoid the problem in the future, if you are adding hot peppers for the heat, is to make a bouquet garni (usually a bundle of herbs tied together in cheesecloth, but the same technique works for peppers), and remove it when the desired heat level is reached.
Always remove the seeds and ribs of peppers as well. This reduces the heat in the dish, and may reduce bitterness with some peppers.
Roasting the peppers first also helps to cool the fire to manageable levels, helping to develop the natural sugars in the fruit.
The secret though is to experiment. Try different techniques to find what works for you for each dish.