Posted on July 18, 2013 @ 01:17:00 PM by Paul Meagher
In my last blog, I discussed some useful probability distributions for representing our uncertainty about a parameter; the uniform and the triangular distributions.
Our uncertainty about a parameter θ such as "the price of gas next week" can be represented using a uniform distribution where the gas price could be anywhere between some low estimate and some high estimate of the price next week. If we also want to hazard a guess as to the most likely value, then we would be using a triangular distribution to represent our uncertainty about the price of gas.
There are other simple techniques for eliciting a probability distribution to represent our uncertainty about a parameter. In today's blog I want to discuss a simple technique called "Merit Scoring".
The Future Price of Corn
The easiest way to explain this technique is if you look at the table below.
Corn Price (per bushel)  Merit Score 
$4.25  ? 
$4.50  ? 
$4.75  ? 
$5.00  ? 
$5.25  ? 
The table has future corn prices ranging from $4.50 to $5.50 per bushel (see quotecorn.com for current price). Now, I might ask you to assign a relative merit score to each price point in this range. A merit score can range between, say, 1
and 10. If you assign a merit score of 1 to a price point, that means you think the future price will not be nearest to that price  the price estimate has low merit. Conversely, a merit score of 10 means that you think the future price will be nearest to that price  the price estimate has high merit. My merit scores for the price of corn on Sept 1, 2013, looks like this.
Corn Price (per bushel)  Merit Score 
$4.25  1 
$4.50  5 
$4.75  10 
$5.00  8 
$5.25  3 
In this example, we are not directly assigning a probability to each possible price point. Instead we are supplying a merit score to each possible price point. We can easily convert each merit score to a corresponding probability by summing all the merit scores and then dividing each merit score by this sum. The result is a probability assignment for each price point with probabilities for each price point summing to 1. This gives as a probability distribution for our parameter which is the price of corn on Sept 1, 2013.
To demonstrate how merit scores can be converted to probabilities and how this forms a probability distribution I have devised a PHPbased script that shows how the calculation is done, what the calculated price probabilities are, and that these probabilities sum to 1.
<?php
/** * @script merit_scoring.php * @author Paul Meagher * @purpose: Convert merit scores to a probability distribution. */
// Enter merit labels and merit scores here.
$merit_scores = array( "4.25"=>1, "4.50"=>5, "4.75"=>10, "5.00"=>8, "5.25"=>3 );
// Compute sum of scores so we can compute probabilities below.
$merit_sum = array_sum($merit_scores);
// Compute the corresponding probability of each label.
foreach($merit_scores AS $merit_label=>$merit_score) $prob_dist[$merit_label] = $merit_score/$merit_sum;
// Dump the contents of the $prob_dist array to the screen.
echo "<pre>"; print_r($prob_dist); echo "</pre>";
// Verify that the sum of each probability is 1
$total_prob = array_sum($prob_dist);
echo "Total probability is $total_prob";
/* Output:
Array ( [4.25] => 0.037037037037037 [4.50] => 0.18518518518519 [4.75] => 0.37037037037037 [5.00] => 0.2962962962963 [5.25] => 0.11111111111111 )
Total probability is 1 */
?>
Conclusions
The merit scoring technique and script can be used to estimate a probability distribution for any parameter that interests you. One limitation of this technique is that it is discrete in nature so can't give you probabilties for prices that might fall between two price points (e.g., $4.85). This may be of concern if you think you should be trying to estimate the future price of corn with more resolution (e.g., 10 cent increments) and/or the daily variability in corn prices is not that high. The daily price of corn is actually quite high so being correct to within 25 cents might be a good goal for your predictions.
