|
||
|
||
There are two types of domain name appraisers, designated here as type “1” and type “0,” with the former being appraisers who rely on a scientific approach. A large number of domain owners use the services of type “0”—the nonscientific—or do the appraisal themselves. Approaches used by scientific appraisers include regression-type statistical modeling, discounted cash-flow analysis, and reliance on the Law of Large Numbers.
This post looks at some of the typical erroneous arguments against taking a statistical approach and provides an example from law to shed light on some success stories with statistical models that have changed people’s minds. For a concrete example, we use a study on predicting Supreme Court decision-making.
Appraisal vs. Valuation
“Appraisal” and “valuation” are typically used interchangeably. Although they are both based on opinion, whether that of an individual or group, a valuation is different in that other researchers can replicate it. Nevertheless, this does not necessarily imply that appraisals cannot provide verifiable predictive accuracy or are inferior to valuation models.
Supreme Court Decision Making Study
The study1 compares the predictive accuracy of legal experts to statistical regression. The selected experts for the study include scholars (five of whom had been law school deans), practitioners, and pundits. The regression model used only six factors: (1) the circuit court of origin; (2) the issue area of the case; (3) the type of petitioner; (4) the ideological direction of the lower court ruling; (5) the type of respondent; and (6) whether the petitioner argued that a law or practice is unconstitutional. The model’s dataset was composed of 628 cases previously decided by the nine justices.
The regression model predicted 75% of the affirm/reverse results correctly, while the legal experts collectively got 59.1% right. The model also predicted Justice O’Connor’s vote (a swing vote) correctly 70% of the time, while the experts’ success rate was only 61%.
Common Criticism of Statistical Models
Unjustifiable skepticism over the use of statistical models for prediction is not confined to domain name valuation. Other fields include law, wine rating, sports, and medicine, to name a few.2
Common themes voiced by the skeptics include too many variables, only “experts” can do it, not enough data, and confusing statistical results. Below I address each of these issues separately.
1. Too many variables. This criticism does not refer to situations where there are too many variables to estimate compared to observations. Rather, the argument maintains that there are too many hard-to-quantify variables and that not all variables have an impact on all domain names—i.e., there is a large number of domain name-specific factors. To counter this argument, the Supreme Court case above demonstrates that six predictors, a relatively small number, perform better than the experts.
2. Only “experts” who have buying and selling experience can appraise. The Supreme Court study also provides a counterexample using regressions. Moreover, in general, by aggregating opinions, irrespective of expertise, one can come up with a robust value estimate. The latter approach is based on the Law of Large Numbers3 instead of regressions.
3. Not enough data. A significant amount of data on sale prices is publicly available, although the publicly available data does becomes thinner for domains sold for greater than $250,000. Nevertheless, there are sources of data, other than sales prices, that can shed even more information on domain name values.4
4. Statistical results are confusing. The group voicing this particular complaint does believe in the power of statistics, but they hear contradictory messages and are left to wonder who is right and who is wrong. “Coffee is good for you,” “Coffee is bad for you,” “Domain guru says appraisals are useless,” “Domain guru says statistical appraisals are, in general, the most robust.” Who should you believe?
Unfortunately, there is no easy way to decide, especially when you are not an expert in the field. One solution is what Andy Grove, the chairman of Intel, did when he had to decide on the method of treatment for his cancer. He dug into the relevant literature to better understand it. Obviously, not everyone has the learning ability, time, or desire to follow such an approach. Although a domain name’s appraisal is not a matter of life and death, you want to make sure that you are not taken for a ride. Thus, at a minimum, you need to be aware of some common sale pitches and their weaknesses:
a. “We have more stats and data.” Ask them how do they know that they have more stats than their competitors and how much difference in value precision does their additional data lead to?
b. If they believe that the length of the domain name is a strong predictor of value, ask them why their belief contradicts other statistical studies.5
c. When someone tells you “I believe company X is the best appraiser,” ask them why. Would that be based on statistical prediction accuracy or the closeness of the appraisals to the recommender’s estimates?
Is Domain Appraisal Worth It?
Standard appraisal fees range from zero to about $40 per domain name. If you have a domain name that is a non-dictionary key word, then statistical models might not be the right approach. On the other hand, for a domain that is a combination of dictionary key words, a fast check of the frequency of the occurrence of the key words and the corresponding number of Ad Sponsors on Google (in addition to checking the domain’s extension and traffic volume) can give you an idea of whether it is worth spending the money on an appraisal and whether a more accurate appraisal is worth the extra cost.
References:
For additional references on domain name appraisal, click here.
1 For a condensed version of the study, see Andrew D. Martin, et al., “Competing Approaches to Predicting Supreme Court Decision Making,” Perspectives on Politics, 2, vol. 4, 2004, pp. 761–767; available here [PDF].
2 Ian Ayres, Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart, Bantam, 2007.
3 Such an approach has been adopted by The Wisdom Of Domainers.
4 See Alex Tajirian, “Food for Thought: Appraisal Dataset,” DomainMart.
5 See, for example, Alex Tajirian, “Length of Domain Name Is Irrelevant!,” DomainMart.
Sponsored byVerisign
Sponsored byWhoisXML API
Sponsored byRadix
Sponsored byCSC
Sponsored byIPv4.Global
Sponsored byVerisign
Sponsored byDNIB.com
A World-Renowned Source for Internet Developments. Serving Since 2002.
Caveat emptor. Since we are still in the begininng stages of, or as some people say, living the equivalent of the Wild West - Gold Rush with respect to domain name buying, you cannot value or appraise any domain name. Yes, there are ways to approximate value by analyzing traffic and ranking yet the true value is in what the domain name “owner” places on it and what the perspective buyer is willing to pay. Its all about vision and feelings, not data! Using mathematical equations or any type of formula to assess the value of a domain name is purely a waste of time. Don’t kid yourself or others, an appraisal of a domain name is simply an opinion with very little factual real life data. As an Attorney who missed an opportunity to obtain http://www.BocaRatonLawoffice.com years ago for $500.00 because I had it appraised at 150.00, I also used an online service to value or appraise http://www.acefla.com, a private investigation company that is for sale, and was told its worth 175,000? Go figure? JM, West palm Beach, FL
Jaysen,
Thanks for pointing out the “newness” of domain names, another erroneous argument, which I missed in my post.
Below are three plausible interpretations of newness and the associated errors:
1. Not enough data to conduct meaningful statistical tests. Not true, as noted in the post.
2. Enough data, but it is pure noise, i.e., no statistically significant patterns in the data. Not true, as noted in the post, statistical data mining techniques suggest otherwise.
3. A domain name’s best use is not yet established. To a certain degree this is true, as in a large number of inventions including the birth of the PC and fax, to name just a few. For domain names, the public discourse includes development, e-commerce, and leasing as use options. Nevertheless, if an entrepreneur finds a new value-adding use, it does not imply that a previously valid appraisal was meaningless at the time it was conducted.
Just because certain arguments didn’t work for one example doesn’t mean they do not work for another. Domain valuation, is predicting it really the same as supreme court decisions?
I would LOVE to see a strong model able to predict values well. Even with the samples I have worked with getting an R^2 of 50% is very hard (i have only got to ~48). There is almost 2 markets to be considered, reseller and end user. Just the way buyers value domains is done at two different levels alone which throws off accuracy of most models.
Since we are all aware you provide appraisals and your models, care to share how strong a model you’ve developed? I would be interested in R^2, and N and how many variables (don’t need to reveal them if you don’t want).
You raise interesting issues. However,
1. The objectives of the post are two fold:
(a) To provide arguments for the viability of statistical methodology for estimating the transactional (ie, not for IRS, trademark, or antitrust considerations) value of a domain name, and
(b) To outline some erroneous arguments regarding the use of scientific methods to value domain names.
Thus, the post is not about the use of a specific statistical technique or a specific appraiser.
2. R^2
(a) Given any dataset, one can construct a regression model with an R^2 of 100%. Thus, it is not necessarily a good measure of goodness-of-fit.
(b) In some statistical techniques, such as regression-trees, R^ is not used as a criterion for selecting viable independent variables and thus, it is not used as a measure of goodness-of-fit.
(c) As the post is related to the use of statistical techniques to predict value rather than to estimate some parameters, a better goodness-of-fit criterion is the estimated model’s predictive power. This can be achieved, for example, by dividing the total sample of observations into two mutually exclusive groups: one for estimation and the other for determining its predictive power.
3. Increasing the number of explanatory variables does not necessarily improve the goodness-of-fit, even if one were to use R^2 as a measure.
Thanks for responding, but I think you have dodged the questions.
You believe it is viable, I would like to see what sort of approach you would use or do use and how accurate a model you can actually create. I have tried personally and I don’t think you can create an accurate statistical model, at least not a strong one. You say R^2 may not be a good measure of fit because you can make any r^2 100 given a data set. Even if you did this, when you use the model to predict with an R^2 of 100 it should be extremely accurate, which would be a good test of whether you just manipulated the data to suit your needs or actually created a good model, no? lastly, I am aware increasing the number of variables doesn’t always improve goodness of fit, there were a lot of variables I tried that made the model worse. However, it is interesting to see how many factors really play a role and can be measured in a statistical model. Would you care to answer any of my original questions now?
Glad that you are interested in the topic. I am not trying to dodge, but I would like to keep the discussion focused on the post and the issues you raised in your reply, which I address below.
It would be constructive if we start with what you mean by accurate.
What constitutes strong vs. weak?
The answer is no! In practice, you will never get 100% R^2, because there is always “measurement error.” An R^2 of 100% can be achieved, for example, by running the estimated line through every observation. However, the resulting model would very likely be practically useless in prediction and thus, such a perfect-fit model defeats the purpose of statistical estimation. Another example of useless perfect fit is running a regression of “right shoe” on “left shoe.” Both examples do not involve manipulating or massaging the data.
We agree that increasing the number of variables does not necessarily improve the fit. Thus, in general, the number of significant variables is irrelevant. Moreover, in practice, we find that certain functional forms of the product of “click volume” and CPC have better predictive power than using them separately. Furthermore, due to the nature of regression trees, the number of variables for each cluster of similar domain names is not necessarily the same. Hence, the number of variables used in the estimated model does not provide any useful information.