California Investment Network


Recent Blog


Pitching Help Desk


Testimonials

"Joined, submitted, we're moving forward. Excellent site, thanks again... "
Steve Smith - EquipmentFX

 BLOG >> Recent

Accurate Classification of Startup Success [Bayesian Inference
Posted on April 4, 2013 @ 11:49:00 PM by Paul Meagher

In my last blog introducing a classification framework for Bayesian Angel Investing, I discussed a php-based software class called ClassifierDiagnostics.php. I showed how you enter bivariate data points into it and the type of output it displays. I didn't go into much detail on what the output is telling us. Today I will go into some more detail on what the output is telling us and start to give some indication as to why it is important if you want to be a successful Bayesian Angel Investor.

One way to formulate the problem of Bayesian Angel Investing is as a classification problem where an Investor is trying to asign a probability to whether a startup belongs to the class of "Successful" (S) companies or "Unsuccessful" (U) companies. One way to do this would be to just rely upon the prior odds of a startup being successful or not. You would not conditionalize the probability assignments (e.g., P(S) = θ1, P(U) = θ2) on information about the start up (e.g., P(S|I) = θ3), just the fact that they are a startup and the historical probabilities that a startup will be unsuccessful or successful. This is more difficult than it sounds because the success of a startup is already conditionalized insofar as we have to delimit the scope of the concept "startup" in some way in order to measure the probabilities of success or not. So let us say we will look at startups confined to some region near the Investor's place of residence - the state or province level statistics on startup success.

Can you use the startup success statistics, your "priors", to make successful investment decisions? My guess is that the rate of success for startups in your region is below 50% so if the probability of any given startup being successful is below 50% it is unlikely you will ever invest. You would have to invest randomly according to a "priors only" strategy (i.e., P(S) = θ1 AND P(U) = θ2) and that would produce losses.

To get more levarage on making good angel investments, you will need to incorporate information about the startup in your classification decision regarding the likely success or not of the startup. You will want to identify types of information that have good diagnostic value in classifying startups into bins labelled Successful (S) and Unsuccessful (U). In the example I provided yesterday I suggested that you could use your evaluation of their business plan as a good indicator of whether the startup might succeed or not. If the business plan addresses enough of your checklist of concerns, then you will assign the business plan a "Pass" value (1), otherwise you assign the business plan a "Fail" value (0). The question then becomes whether our pass/fail assignments can be used to successfully distinguish between successful and unsuccessful startups. In other words, how diagnostic is a good business plan of being a successful startup?

In my last blog, I entered observations of 4 startups into my classifier diagnostics program. Each observation consisted of two values, a value specifying whether the business plan passed (1) or failed (0), and a value specifying whether the startup eventually succeeded (1) or failed (0) in their enterprise. When I entered the data into my classifier diagnostics program it generated the output below. I have removed some of the statistics being reported because I want to focus on the foundational concepts in diagnostic problem solving.

Successful Company
Yes No
Business Plan Pass 2
(TP)
0
(FP)
Fail 1
(FN)
1
(TN)
Successful Company
Yes No
Business Plan Pass 0.67
(TP)
0.00
(FP)
Fail 0.33
(FN)
1.00
(TN)
Test Sensitivity (TP) 0.67
False Alarm Rate (FP) 0.00
Miss Rate (FN) 0.33
Test Specificity (TN) 1.00

One critical observation to make about this data is that business plan quality is not a perfect test for classifying startups as successful or unsuccessful. The most grievous error is the case where a startup had a failing business plan but ended up being successful (example of a "miss" or false negative). The test "missed" the correct classification. Because we have such a low sample size, 4 startups, this one error throws our percentages around quite a bit.

What we are looking for in a good test of statup success is one that has high Test Sensivity and high Test Specificity. Test Sensitivity measures the proportion of actual positives which are correctly identified as such. Test specificity measures the proportion of actual negatives which are correctly identified as such. In real life, test sensitivity and specificity are seldom 1, so we have to figure out how we will cope with false alarms (negative instances identified as positive instances) and misses (positives instances identified as negative instances). Rise averse angel investors will likely be more worried about false alarms than misses because in the case of a false alarm you could invest in an unsuccessful company and lose money whereas in the case of a miss you will not have invested in a successful company but will have at least retained your money.

One way to proceed towards becoming a Bayesian Angel Investor is to do some diagnostic work and figure out what types of tests are the best to use in order to classify startups into those who will succeed or not. When evaluating tests to use, you should examine the diagnostic accuracy of your tests using some of the metrics provided above (Test Sensitivity, False Alarm Rate, Miss Rate, Test Specificity). Bayesian Angel Investing likely taps into the same problem solving skills as a doctor who must diagnose whether a patient has cancer or not. They will order up a series of tests (often binary scored) and make a diagnosis, or, if matters are still unclear, order up more tests (e.g., scans, probes, incisions, etc...) so that they can achieve more confidence in their decision making.

Permalink 

 Archive 
 

Archive


 November 2023 [1]
 June 2023 [1]
 May 2023 [1]
 April 2023 [1]
 March 2023 [6]
 February 2023 [1]
 November 2022 [2]
 October 2022 [2]
 August 2022 [2]
 May 2022 [2]
 April 2022 [4]
 March 2022 [1]
 February 2022 [1]
 January 2022 [2]
 December 2021 [1]
 November 2021 [2]
 October 2021 [1]
 July 2021 [1]
 June 2021 [1]
 May 2021 [3]
 April 2021 [3]
 March 2021 [4]
 February 2021 [1]
 January 2021 [1]
 December 2020 [2]
 November 2020 [1]
 August 2020 [1]
 June 2020 [4]
 May 2020 [1]
 April 2020 [2]
 March 2020 [2]
 February 2020 [1]
 January 2020 [2]
 December 2019 [1]
 November 2019 [2]
 October 2019 [2]
 September 2019 [1]
 July 2019 [1]
 June 2019 [2]
 May 2019 [3]
 April 2019 [5]
 March 2019 [4]
 February 2019 [3]
 January 2019 [3]
 December 2018 [4]
 November 2018 [2]
 September 2018 [2]
 August 2018 [1]
 July 2018 [1]
 June 2018 [1]
 May 2018 [5]
 April 2018 [4]
 March 2018 [2]
 February 2018 [4]
 January 2018 [4]
 December 2017 [2]
 November 2017 [6]
 October 2017 [6]
 September 2017 [6]
 August 2017 [2]
 July 2017 [2]
 June 2017 [5]
 May 2017 [7]
 April 2017 [6]
 March 2017 [8]
 February 2017 [7]
 January 2017 [9]
 December 2016 [7]
 November 2016 [7]
 October 2016 [5]
 September 2016 [5]
 August 2016 [4]
 July 2016 [6]
 June 2016 [5]
 May 2016 [10]
 April 2016 [12]
 March 2016 [10]
 February 2016 [11]
 January 2016 [12]
 December 2015 [6]
 November 2015 [8]
 October 2015 [12]
 September 2015 [10]
 August 2015 [14]
 July 2015 [9]
 June 2015 [9]
 May 2015 [10]
 April 2015 [9]
 March 2015 [8]
 February 2015 [8]
 January 2015 [5]
 December 2014 [11]
 November 2014 [10]
 October 2014 [10]
 September 2014 [8]
 August 2014 [7]
 July 2014 [5]
 June 2014 [7]
 May 2014 [6]
 April 2014 [3]
 March 2014 [8]
 February 2014 [6]
 January 2014 [5]
 December 2013 [5]
 November 2013 [3]
 October 2013 [4]
 September 2013 [11]
 August 2013 [4]
 July 2013 [8]
 June 2013 [10]
 May 2013 [14]
 April 2013 [12]
 March 2013 [11]
 February 2013 [19]
 January 2013 [20]
 December 2012 [5]
 November 2012 [1]
 October 2012 [3]
 September 2012 [1]
 August 2012 [1]
 July 2012 [1]
 June 2012 [2]


Categories


 Agriculture [77]
 Bayesian Inference [14]
 Books [18]
 Business Models [24]
 Causal Inference [2]
 Creativity [7]
 Decision Making [17]
 Decision Trees [8]
 Definitions [1]
 Design [38]
 Eco-Green [4]
 Economics [14]
 Education [10]
 Energy [0]
 Entrepreneurship [74]
 Events [7]
 Farming [21]
 Finance [30]
 Future [15]
 Growth [19]
 Investing [25]
 Lean Startup [10]
 Leisure [5]
 Lens Model [9]
 Making [1]
 Management [12]
 Motivation [3]
 Nature [22]
 Patents & Trademarks [1]
 Permaculture [36]
 Psychology [2]
 Real Estate [5]
 Robots [1]
 Selling [12]
 Site News [17]
 Startups [12]
 Statistics [3]
 Systems Thinking [3]
 Trends [11]
 Useful Links [3]
 Valuation [1]
 Venture Capital [5]
 Video [2]
 Writing [2]