Latin Hypercube vs. Monte Carlo Sampling

A copula structure generated using RLHS.

In a recent post on Linked In, David Vose argues that the advantages of Latin Hypercube sampling (LHS) over Monte Carlo are so minimal that "LHS does not deserve a place in modern simulation software." [1]  He makes some interesting points, yet products like Analytica and Crystal Ball still provide LHS and even offer it as their default method. Why? Are we, the makers of these simulation products naïve? As the lead architect of Analytica for two decades, I've explored this question in detail. I've compared the performance of LHS vs. Monte Carlo on hundreds of real-world models. And I've concluded that yes —it does make sense to keep Latin Hypercube as the default method. Let me explain why I disagree with David Vose on some issues and agree with him on others. Several of his complaints are specific to Crystal Ball or @Risk and don't apply to Analytica. Then I'll add some key insights garnered from my own experience. What is Latin Hypercube Sampling? First some background. (Feel … [Read more...]

Combination of Assessed Distributions

Future Population

Suppose you need information about the future population Metropolia City in the year 2020.  So you solicit two experts to provide assessments of this quantity and provide their expert judgments in the form of distributions as shown. Which assessment is better?  Who is right?  How can you combine these to arrive at a combined distribution that embodies the combined knowledge of both experts? The experts' distributions in this example are LogNormal(100K, 1.2), and LogNormal(90K,2). Calibration Arguably the most important criterion for judging a subjective assessment is for the source of that assessment (i.e., the expert) to be well-calibrated.   Calibration is a property of the assessor, not a property of a single assessment, and reflects the accuracy to which an expert can quantify his degree of uncertainty.   Being well-calibrated reflects the idea that over the course of many assessments, his assessed probabilities correspond to the empirical frequency of occurrence.  … [Read more...]