Model Risk Last revised: 03/02/02
Links | Ask Dr. Risk! | If Only You Had Asked Dr. Risk! | Presentations | Terms & Definitions | Puzzles | Bibliography
Ask Dr. Risk
7/28/00 Clueless in Chicago (7/28/00)
Dear Dr. Risk – We've got a pricing model that we use on a daily basis, hundreds -- sometimes thousands -- of times. Our regulator is pressing us for details of the model. Our problem is that we don't have any documentation to give. We know the model's good, 'cause we're still in business, but we can't document the fact. Can you help us figure out which model we're using. – Chicago
Dear Chicago – Your bank is certainly not the only one using an undocumented "black box" to price options with millions of dollars of notional value. Once, Dr. Risk was faced with the problem of documenting a pricing application for which the bank had no write-up. They didn't even have a name for the model, such as Black-Scholes.
"No problem," said Dr. Risk. "Let me see the source code. I don't care what language."
A few days later, the liaison had some bad news: "The source code isn't on the network. They were running out of disk space one day, a few years ago, and nobody had accessed the file for a few years, so they moved it onto magnetic tape and stuck the tape into an archive. We'll have to get it out of the archive."
A week later, the liaison had some bad news: "The archive was getting full, and nobody had asked for that tape after five years, so somebody chucked it."
"No problem," said Dr. Risk. "Let me talk to the developer."
A few days later, the liaison had some bad news: "The developer left the bank a few years ago for a new job. Then he left that job. We can't find him. A guy who worked with the developer is still with the bank. He's not answering his phone. Maybe he's on vacation. We'll ask him and get back to you."
A few days later, the liaison had some bad news: "The assistant didn't answer his phone, because he's recovering from a heart attack. He's supposed to rest and may be taking early retirement. I don't think he's going to be available. Got any ideas?"
Dr. Risk said, "Let's see if I can match the output with something in my library. We don't have a whole lot of right ways to go, here."
Turns out the model wasn't difficult to match. It would be nice to see the guts of the application, to make sure it doesn't contain a Trojan horse, but that's not going to happen.
In your case, if you like, Dr. Risk can try to reverse engineer your application. With a little bit of luck we'll find a perfect match for your app in our extensive library of models. Let us know if you want us to pursue this as a consulting project. – Dr. Risk
6/28/00 Will the real Garman-Kohlhagen model please stand up? (6/28/00)
Dear Dr. Risk – Checking
a great number of online currency option calculators lead me to think that there
are at least two different Garman-Kohlhagen formulas or perhaps a different use
of option parameters.
To be more Specific, I priced a Call USD Put JPY option with the following
parameters:
Spot = 106
Strike = 115
Vol = 13.6453%
USD Depo = 5%
JPY Depo = 1%
T = 0.505555556 (182 days to expiry)
Many calculators (and yours) showed a result of 0.800383, while others reached
0.788688
I will be very happy if you show me the way – Moses
Dear Moses – This is a great question, because it illustrates how difficult it is to use even the most basic and commonplace calculators. It’s not just you, Dr. Risk pulled out what little hair he had a few years ago when testing eight commercial option pricing calculators against each other and his own calculators. Every vendor had his own, peculiar conventions for time, interest rates, and Greeks. While they overlapped some with the competitors’ conventions, the unique parts were sometimes temporarily baffling.
Dr. Risk suspects that your entire problem hinges on different ways to convert days into years. You chose to use a U.S. banker's year with 360 days, so you computed time to expiration of 0.505555556 = 182/360 years, whence the call’s value is 0.800383. However, if you had joined Japanese bankers and most normal people by assuming that a year has 365 days, you would have gotten T = 0.4986301 = 182/365 years, whence C = 0.7883242. That would be close enough to your answer for government work, but you work for a bank, so ...
Some option model developers have assumed that a year has 365.25 days, and others let some years have 365 and other years have 366. Industry standard calculators go further with their assumptions and conventions. The USD money market rates are quoted as actual/360, while Japanese money market rates are actual/365. This affects the conversion of your quoted deposit rates into the continuously compounded rates that academic models tend to use.
Then there's the issue of "volatility time" versus "discounting time". Volatility is relevant for the entire period from pricing until expiration, because that's when the market can receive information that affects a price. The price reflects discounting over a different period. The usual academic formulas assume immediate settlement at purchase and exercise. Reality is different. Even with T+1 settlement, a "spot" price is really a forward price -- one day forward. For an option with no "dividend" or other cash distribution, the forward price will exceed the spot price by one day's carry. Then, at exercise it takes a while to get the payoff, and theory calls for payoff as a function of spot price, but the market quotes a forward price. Many professionals concern themselves with this sort of minutia, because ignoring it can be expensive.
Moses, with all these issues to resolve, Dr. Risk hopes that you don't end up confused and wandering for 40 years (say) in the Sinai. – Dr. Risk
If Only You Had Asked Dr. Risk!
Presentations
Recent Past Presentations about Model Risk
1999 March 22-23. Effectively Managing & Mitigating Model Risk. In New York at the Roosevelt Hotel. Sponsor: Institute for International Research. Contact: IIR Registration. by email or telephone at 888-670-8200. Download brochure. Hear Morton Allen, Peter Carr, Oren Cheyette, Kevin Cassidy, Terry Chong, Kenneth Cooney, David Green, Bruce Jurin, Michael Kamal, Sean Keenan, Michel McCarthy, Elliott Noma, Lorna Ness, Jerzy Pawlowski, Curt Randall, Jayanthi Sankaran, Richard Skora, Jorge Sobehart, Nassim Taleb, Charles Wang, Paul Watterson, Edward Weinberger, and Steven Whiting talk about model risk in many forms from industry experience.
William Margrabe's Past Presentations on Models and Model Risk
1999 May. "Advanced Derivatives." A federal government regulatory agency.
1999 April 30. "The Wand and the Wizard" and "Beware of Greeks". How to avoid the most common "knucklehead mistakes" with the most common option pricing models. In Chicago. Sponsor: University of Chicago, Mathematical Finance Seminar.
1999 March 22-23. Chairperson, Effectively Managing & Mitigating Model Risk. In New York at the Roosevelt Hotel. Sponsor: Institute for International Research. Contact: IIR Registration. by email or telephone at 888-670-8200. Download brochure.
- 1998 September 18. "Models and Model Risk." International Association of Financial Engineers. Conference and Annual Meeting. New York, NY.
- 1998 August 10-14. "Advanced Derivatives." A federal government regulatory agency.
- 1997 December 1-4. "Advanced Derivatives." A federal government regulatory agency.
1997 October 6. "How Do I Validate Thee? Let Me Count the Ways." (Managing Model Risk.) Institute for International Research. North American Summit - Risk ’97. Miami, FL.
1996 December 5. "The Wand and the Wizard" (The significance of major and minor variations in numerical methods). Courant Institute of Mathematical Sciences.
Terms and Definitions
- "discrete" dividends
- Definition: Dividends that come in discrete lumps on specific dates, as opposed to a continuous dividend yield.
- Example: Shares in ABC Corp. might pay a discrete dividend of $1.50 on the 15th of each month to the holder of record as of the first of the month.
- Pricing: A binomial model or finite difference model with steps in log of spot price cannot allow for discrete dividends in a straight forward way. This causes a problem in pricing American equity and equity index options with standard methods. According to theory and common sense, early exercise comes only just before a dividend payment, but a pricing algorithm with dividend yields (rather than discrete dividends) may have early exercise at any time.
- Model Risk
- Definition: The risk of losing money and/or office, because a model fails to match reality sufficiently well, or otherwise deliver the required results.
- Example: A controller marks a trader's positions by taking volatility directly from transaction prices in caplets. Unfortunately, he gets yield volatilities, whereas he needs price volatilities.
- Application: The internal auditor used three different models to value the exotic option and got three, wildly divergent values. The trader's value match the highest of the three. The internal auditor began to worry about model risk.
- Pricing: Putting a dollar figure on a particular risk is difficult, because it depends on the behavior of the parties using the model. For example, Kidder, Peabody's Government Trader system contained a tiny mistake that might have been insignificant, if traders had not done forward settlement out beyond corporate settlement. However, Joseph Jett did forward Treasury strips and recons that settled months into the future, which magnified the significance of the problem.
- Risk Management: The best way to manage model risk is to review models, document their weaknesses, and make sure that nobody uses the models in dangerous ways.
- Comment: Every model is a simplification of reality. If you push it far enough, it will fail. The challenge is to do that in a controlled environment that doesn't produce real losses.
Puzzles
Dr. Risk discusses issues like these in his classes on model risk.
The Sawtooth that Bites
You are looking in a popular weekly derivatives publication at the forward curve for USD LIBOR. You notice it has a distinctly jagged look to it. In particular, it has saw teeth with peaks around five, seven, and ten years. Why are they there? Is it because of market conditions, or a characteristic of someone's models? How might a trader create false profit by taking advantage of this situation?
A Matter of Opinion?
One of your option traders is making a huge amount of money, according to official firm books and records. His controller is concerned that something might be amiss, because he can't find vols, except for the at-the-money options. The trader says they shouldn't worry, because the deep OTM options are worthless and the deep ITM options trade like forwards. Who's right?
Links
- Derman, Emanuel. "The Future of Modelling." Risk (December 1997).
- Derman, Emanuel. "Model Risk." Goldman Sachs Quantitative Strategies, Research Notes (April 1996).
- Elliot, Margaret . "Controlling Model Risk." Derivatives Strategy (June 1997).
- Tanenbaum, Richard. "Avoiding Software Time Bombs."
Bibliography
- Beder, Tanya Styblo; Minnich, Michael; Shen, Hubert; and Standon, Jodi. "Vignettes on VAR." Journal of Financial Engineering 7 (3/4), September / December 1998, pp. 289-309.
- Bliss, Robert R. and Smith, David C. "The elasticity of interest rate volatility: Chan, Karolyi, Longstaff, and Sanders revisited." Journal of Risk 1 (1), Fall 1998, pp. 21-46.
- Chan, K.C.; Karolyi, G.A.; Longstaff, F.A.; and Sanders, A.B. "An empirical investigation of alternative models of the short-term interest rate. Journal of Finance 47 (1992), pp. 1209-1227.
- Crouhy, Michel; Galai, Dan; and Mark, Robert. "Model Risk." Journal of Financial Engineering 7 (3/4), September/December 1998, pp. 267-288.
- Derman, Emanuel. "Valuing Models and Modeling Value." Journal of Portfolio Management (Spring 1996), pp. 106-114.
- Derman, Emanuel. "Model Risk." Goldman Sachs Quantitative Strategies, Research Notes (April 1996).
- Derman, Emanuel. "The Future of Modelling." Risk (December 1997).
- Elliot, Margaret. "Controlling Model Risk." Derivatives Strategy (June 1997).
- Fallon, William. "Rogue models and model cops." Risk (September 1998), pp. 24-31.
- Jett, Joseph (with Sabra Chartrand). Black and White on Wall Street. New York: William Morrow, 1999.
- Locke, Jane. "Credit Check." Risk (September 1998), pp. 40-44.
Books
 |
We can learn a great deal from this well-written, but depressing book – for what it says about Joseph Jett's experiences with race relations in the U.S., about the way Kidder, Peabody operated in early 1990s, and about model risk. The book is fascinating, well-written, and persuasive, and nobody comes out smelling like a rose, except his attorneys. Clearly his book is self-interested, but it makes more sense than press reports of the affair. That is one more sad thing. [more] – Dr. Risk |
Our other web sites:
www.FreeOption
Pricing.com
Free option pricing calculators from here and around the world.
www.RiskManagement
Digest.com
Summaries
of the best articles
from the best publications
in the risk management trade press.
www.Derivatives
Digest.com
Summaries
of the best articles from the best publications
in the derivatives trade press.
www.AskDrRisk
.com
Answers to your questions about Investment,
Risk Management,
Derivatives, and
Financial Engineering