Garbage in, garbage out trampled by Moore's law

[Robert Myers]

Told you so.

http://arstechnica.com/news.ars/post/20081215-axioms-downturns-and-a-global-computer-crash.html

Robert.

 

[Robert Myers]

global-computer-crash.html

So, you think that the re-regulation of the financial system will involve
mandatory error bounds on stock value predictions?  ;-)

The point is that such error bounds aren't possible, even in theory.
Everyone's assumptions about how many sigma out their model breaks
depends on everyone else's assumptions.

The question is whether very large computers really can be used for
all but the most trivial and transparent of tasks: animation, say, or
physics modeling, or gene sequencing. Tasks that are computationally
intensive but fundamentally simple and repetitive. The Monte Carlo
simulations of financial engineers look(ed) like an ideal candidate.

The jury may still be out, but for a long time I've been leaning
toward, "No, they can't be."

Robert.

 

[Del Cecchi`]

The point is that such error bounds aren't possible, even in theory.
Everyone's assumptions about how many sigma out their model breaks
depends on everyone else's assumptions.

The question is whether very large computers really can be used for
all but the most trivial and transparent of tasks: animation, say, or
physics modeling, or gene sequencing. Tasks that are computationally
intensive but fundamentally simple and repetitive. The Monte Carlo
simulations of financial engineers look(ed) like an ideal candidate.

The jury may still be out, but for a long time I've been leaning
toward, "No, they can't be."

Robert.

Airplanes and bridges have gone down because of bad assumptions, now we
have an example from finance. The mortgage mess had nothing to do with
fancy monte carlo simulations, unless you think a spreadsheet is fancy,
which I am sure you don't.

The whole mess was basically predicated on the notion that house prices
never decrease, at least simultaneously nationwide.

Although the "genius" that thought selling Credit Default Swaps aka bond
insurance to people who didn't even own the bond was a good idea should
be ashamed.

del

 

[Robert Myers]

Airplanes and bridges have gone down because of bad assumptions, now we
have an example from finance.  The mortgage mess had nothing to do with
fancy monte carlo simulations, unless you think a spreadsheet is fancy,
which I am sure you don't.

The whole mess was basically predicated on the notion that house prices
never decrease, at least simultaneously nationwide.

Although the "genius" that thought selling Credit Default Swaps aka bond
insurance to people who didn't even own the bond was a good idea should
be ashamed.

The problem is much more fundamental than a single faulty assumption:

<quote>

Ritholtz writes, the following about the way in which the quants built
their models:

"As Benoit Mandelbrot, the fractal pioneer who is a longtime
critic of mainstream financial theory, wrote in Scientific American in
1999, established modeling techniques presume falsely that radically
large market shifts are unlikely and that all price changes are
statistically independent; today's fluctuations have nothing to do
with tomorrow's—and one bank's portfolio is unrelated to the next's.
Here is where reality and rocket science diverge."

</quote>

NASA ran into the same difficulties with its fault tree analysis, and
it seems fundamental to me. Not only are there joint probabilities
that are never taken into account, but there is no way even to
estimate those joint probabilities.

How would you predict when there would be a run on Madoff's Ponzi
scheme? How do you estimate the probability that there is a Madoff
out there?

Once Madoff's bogus assets disappeared from balance sheets, it was as
if $50 billion in real wealth had disappeared, triggering still more
asset sales into a distressed market. That is to say, you couldn't
include the extraordinary event in your model and, even if you did, no
one's model takes account of the interdependence of prices (margin
calls-> asset devaluation -> run on the bank -> non-existent assets
exposed -> more margin calls -> more asset devaluation.)

By comparison, the continuum mechanics of airplanes and bridges are
trivial.

Robert.

 

[Bernd Paysan]

global-computer-crash.html

So, you think that the re-regulation of the financial system will involve
mandatory error bounds on stock value predictions? ;-)

I don't know how these models look like, but a model that allows even to
input an "infinite, indefinite, perpetual growth" of some number is already
broken at the model level. This model must be a non-physical model, i.e. one
which does not represent reality, and does not care about boundary
conditions which reality always has.

Models like that are common, even spice models in digital circuit simulation
work like that. You can look at a diode model - it has an exponential
current curve over voltage (plus some resistance and capacitance, and the
current curve is asymmetric with a much higher threshold in reverse
direction). In reality, there's a boundary: The current heats the diode, and
eventually, it melts down, changing everything (after meltdown, there's no
P/N transition anymore, and therefore no diode, just a resistor). There
simply is no unlimited exponential growth.

Furthermore, in reality, noise plays an important role. Electric noise in a
diode just as well as random changes in supply and demand of real goods
(like houses). If your model comes without noise, it can only simulate
stable conditions, where noise will just return to the original state. You
then must find out the phase margin of your simulation, and predict a noise
magnitude, to see if your system really is stable.

 

[Andrew Reilly]

The point is that such error bounds aren't possible, even in theory.

Hence the winky-smiley. Still, I wonder whether there's any correlation
between present financial difficulty and wildness of the "financial"
models being relied-upon, among the denizens of Wall St?

Everyone's assumptions about how many sigma out their model breaks
depends on everyone else's assumptions.

The question is whether very large computers really can be used for all
but the most trivial and transparent of tasks: animation, say, or
physics modeling, or gene sequencing. Tasks that are computationally
intensive but fundamentally simple and repetitive. The Monte Carlo
simulations of financial engineers look(ed) like an ideal candidate.

Luckily there's plenty of interesting problems that fit into those
categories (trivial and transparent) to keep the computers busy. Of
course, there's also a bunch of interesting computation that uses integer
and other exact arithmetics that have no trouble being as elaborate as
you please. Not usually on such a scale, I suppose (databases and
communication and web and so on).

The jury may still be out, but for a long time I've been leaning toward,
"No, they can't be."

That may be so, but (a) can it be legislated that way and (b) if not,
wouldn't you still put your retirement money with the guy who's models/
predictions/gut-hunches are working out the best? When it's the only
game in town...

Cheers,

 

[Anne & Lynn Wheeler]

So, you think that the re-regulation of the financial system will involve
mandatory error bounds on stock value predictions? ;-)

re:
http://www.garlic.com/~lynn/2008s.html#23 Garbage in, garbage out trampled by Moore's law

GAO has been doing database of increasing numbers of financial
restatements of public companies (in spite of SOX). Basically
executives fiddle statements in order to increase bonuses. Later
statements may be restated, but executives don't forfeit bonuses. One of
the worst examples was freddie was fined $400m in 2004 for $10b
statement fiddling/inflation and the CEO replaced ... but allowed to
keep tens (hundred?) of millions. an earlier GAO reference:
http://www.gao.gov/cgi-bin/getrpt?GAO-03-138
2006 GAO reference:
http://www.gao.gov/new.items/d06678.pdf

post from earlier this year (with several additional references)
http://www.garlic.com/~lynn/2008f.html#96

with respect to rating agencies giving triple-A ratings to toxic CDOs,
supposedly SOX required SEC to do something with respect to the rating
agencies ... but there doesn't seem to have been anything besides a
Jan2003 report.

Report on the Role and Function of Credit Rating Agencies in the
Operation of the Securities Markets; As Required by Section 702(b) of
the Sarbanes-Oxley Act of 2002
http://www.sec.gov/news/studies/credratingreport0103.pdf

another reference:
http://www.garlic.com/~lynn/2008s.html#5
and some related items:
http://www.garlic.com/~lynn/2008s.html#9

I would claim that regulation of the financial infrastructure and
insider anti-fraud processes are closely related. this recent post
mentions an early 80s court case involving (silicon valley, computer)
industrial espionage ... and the court effectively required
demonstrating that anti-theft/anti-fraud processes (which were
proportional to the value of the information, in the particular
situation, a couple billion dollars) had to be in place
http://www.garlic.com/~lynn/2008s.html#5 Greed - If greed was the cause of the global meltdown then why does the biz community appoint those who so easily succumb to its temptations?

in the above post, i mentioned that in a 2004 european financial
executive conference, i claimed that SOX was in large part window
dressing.

the analogy (in the industrial espionage court case) was akin to
requiring fences around swimming pools since minors can't be held
responsible for going swimming. given sufficient temptation ... the
court basically assumed everybody would steal something valuable
.... unless there were countermeasures.

Asking why financial regulation is needed is possibly on par with
wondering why banks might use vaults to keep money. The court (in the
particular case from the early 80s claiming billions in damages)
.... bascially wanted, in additon to showing that the information had
been stolen (and used so that there was resulting damages), proof that
there had been anti-theft processes in place (and considered adequate to
protect something worth billions of dollars, aka "security proportional
to risk").

 

[Anne & Lynn Wheeler]

re:
http://www.garlic.com/~lynn/2008s.html#23 Garbage in, garbage out trampled by Moore's law
http://www.garlic.com/~lynn/2008s.html#24 Garbage in, garbage out trampled by Moore's law

and for more of the view requiring regulation:

Corporate Fraud and Misconduct Risks Driven by Pressure to do
'Whatever It Takes'; Fewer episodes reported by companies with ethics
and compliance programs
http://www.financetech.com/news/showArticle.jhtml?articleID=212501185

from above:

Of more than 5,000 U.S. workers polled this summer, 74 percent said
they had personally observed misconduct within their organizations
during the prior 12 months, unchanged from the level reported by KPMG
survey respondents in 2005. Roughly half (46 percent) of respondents
reported that what they observed "could cause a significant loss of
public trust if discovered," a figure that rises to 60 percent among
employees working in the banking and finance industry.

.... snip ...

 

[Robert Myers]

Of more than 5,000 U.S. workers polled this summer, 74 percent said
they had personally observed misconduct within their organizations
during the prior 12 months, unchanged from the level reported by KPMG
survey respondents in 2005. Roughly half (46 percent) of respondents
reported that what they observed "could cause a significant loss of
public trust if discovered," a figure that rises to 60 percent among
employees working in the banking and finance industry.

Perhaps the computers and the PhD's in economics are nothing more than
window dressing, or perhaps even worse: a cover for garden-variety
fraud.

Robert.

 

[Anne & Lynn Wheeler]

Perhaps the computers and the PhD's in economics are nothing more than
window dressing, or perhaps even worse: a cover for garden-variety
fraud.

re:
http://www.garlic.com/~lynn/2008s.html#23 Garbage in, garbage out trampled by Moore's law
http://www.garlic.com/~lynn/2008s.html#24 Garbage in, garbage out trampled by Moore's law
http://www.garlic.com/~lynn/2008s.html#27 Garbage in, garbage out trampled by Moore's law

there is the old line about asking crooks why they rob banks ... and the
answer is: that is where the money is. if overall number is 46% ("could
cause a significant loss of public trust if discovered") and number for
financial is 60%, then the non-financial industry number should be
someplace under 40% ... making financial industry at least 50% worse
than other industries.

recent/similar thread in some linkedin discussions ... where i commented
that SOX (sarbanes-oxley passed in the wake of enron & worldcom) was
more like "window dressing" & some amount of the input fiddling was
a case of "garbage in, garbage out":
http://www.garlic.com/~lynn/2008s.html#9 Blind-sided, again, Why?

as implied in the reference to the (silicon valley, computer related)
industrial espionage litigation from early 80s, the assumption is that
everybody is a crook (given sufficient temptation) and countermeasures
are required that are proportional to risk (and/or proportional to
temptation).