← 返回列表

nominal delivery draft, UNC Charlotte,Prediction and The Future of Cybersecurit

聚合站 SecWiki新闻 入库 2026-11-17T23:41:19
[ nominal delivery draft, UNC Charlotte, 5 October 2016 ]
Prediction and The Future of Cybersecurity
Dan Geer
Hello, All.  With respect, since I wrote the abstract for this talk
I've rethought what is the important thing or things to say.  I had
said that my subject would be "What If we Win?" but the more I
thought about it, the more I realized that that required predictions
of an extraordinary precision and validity.  After some consideration
of this matter, I have decided to speak on a slightly different
topic, one that I think is a necessary pre-condition to be able to
talk about winning the peace after we win the war.  My change is
to that of predicting the future of cybersecurity.
Given the topic I chose, it seems appropriate to start with a triplet
of quotes from others about prediction.
First up is Jeff Hawkins:
  Prediction is not just one of the things your brain does.  It is
  the primary function of the neo-cortex, and the foundation of
  intelligence.
Then there is Milton Friedman:
  The only relevant test of the validity of a hypothesis is comparison
  of prediction with experience.
And, finally, Warren Ellis:
  I try not to get involved in the business of prediction.  It's a
  quick way to look like an idiot.
With those three we have an idea of what prediction is, (1) that
it is a primary function of intelligence, perhaps even the core of
what intelligence is, (2) that, like science in general, claims
that cannot be tested are not worth listening to, and (3) that quite
possibly the surest prediction to make is that in making a prediction
your intelligence will become testable and found to be lacking.
Farmers, weather forecasters, and cybersecurity designers know that
especially well, and perhaps more than all others those three
professions beat their practitioners into a state of humility.  I
actually do two of them, and, as such, let me assure you that these
remarks are presented in all humility, but humility does not mean
timidity.  Rather, it means that when a strongly held belief is
proven wrong, that the humble person changes their mind.  I expect
that I will have to change my mind, perhaps as a result of talking
with you today.
This essay is an outgrowth from, an extension of, the increasing
importance of cybersecurity.  With the humility of which I spoke,
I do not claim that I have the last word.  What I do claim is that
when we speak about cybersecurity policy we are no longer engaging
in some sort of parlor game.  I claim that policy matters are now
the most important matters, that once a topic area, like cybersecurity,
becomes interlaced with nearly every aspect of life for nearly
everybody, the outcome differential between good policies and bad
policies broadens, and the ease of finding answers falls.  As H.L.
Mencken so trenchantly put it, "For every complex problem there is
a solution that is clear, simple, and wrong."
And when do policies fail?  Policies fail when the predictions of
their effect turn out to be wrong but their proponents do not have
the humility to admit it.  Instead, the proponents side with the
predictions they made and insist on ever more vigorous -- some might
say desperate -- measures to make those predictions come true.
Steven Gillon's book, _That's Not What We Meant to Do_, makes this
entirely clear and at length.  The need for prediction *and* for
humility applies to cybersecurity and does so in spades[SG].  David
Wheeler largely restated this for the field of computing by noting
that:
  All problems in computer science can be solved by another level
  of indirection... Except for the problem of too many layers of
  indirection.
In what to me is one of the best books I ever read, Peter Bernstein's
_Against the Gods: The Remarkable Story of Risk_, Bernstein adopts
Elroy Dimson's definition of risk as "that more things can happen
than will"[PB].  Between the size of the event space -- those more
things can happen than will -- and the size of the tools we have
to change what can happen, you have the playing field on which we
operate.
And it is essential to remember that all security technology is
dual use -- it is intent that determines how the technology will
be used which, of course, means that intent is also something to
predict particularly when policy's purpose is to modify what people
want and intend.
To be entirely clear, this essay is written by someone as his own
opinion and not on behalf of anyone else.  It is written without
the supposed benefits of insider information; I hold no Clearance
but am instead informed solely by way of open source intelligence.
This path may be poised to grow easier; if the chief benefit of
having a Clearance is to be able to see into the future a little
further than those without one, then it must follow that as the
pace of change accelerates the difference between how far ahead in
time you can see with a Clearance versus how far ahead you can see
without one will shrink.
The one thing that may make cybersecurity different than all else
is that we have sentient opponents.  The physicist does not.  The
chemist does not.  Not even the economist has sentient opponents.
We do.  What puzzles we have to solve are not drawn from some
generally diminishing store of unsolved puzzles, nor could our
theories completely explain all observable fact thus reducing our
worries and our work to engineering alone.  There is something
different about a search for truth when there isn't any, or at least
any that lasts long enough to exhaustively explore.  There is
something different when what we can detect and from which we can
then infer is even partly under the control of people at cross
purposes to our purposes.
Over a small number of years, the term "data science" has become
commonplace.  It seems first to have been used over 50 years ago,
but the current usage stems most directly from a 1997 lecture by
Jeff Wu with the title "[Does] Statistics = Data Science?"[JW].  Wu
characterized statistical work as a trilogy of data collection,
data modeling, and decision making.  In his lecture's conclusion,
he initiated the modern usage of the term "data science" and advocated
that statistics be renamed data science and statisticians renamed
data scientists.  Those semantics seem to add little clarity to
what the collection, modeling, and use of data provide, but argument
over terminology is a hallmark of how a science develops.
Which brings us to ask whether cybersecurity is, or could become,
a science.  A few months ago I tried to round up some ideas on that
question[NSF], taking as my guide T.S. Kuhn's seminal work, _The
Structure of Scientific Revolutions_[TSK].  The question of whether
cybersecurity is yet a science is a hard one.  I am sorely tempted
to answer the question "Is cybersecurity a science" with "Getting
closer, but not yet" -- to say, in other words, that we are in a
pre-science stage.
With several others, I am one of the expert reviewers for the
National Security Agency's annual "Science of Security" competition
and award[SOS].  Quoting its rationale, "The competition was
established to recognize the current security paper that best
reflects the conduct of good science in the work described.  [Science
of Security] is a broad enterprise, involving both theoretical and
empirical work.  While there can only be one best paper, any one
paper cannot span that full breadth.  Nonetheless, the field is
broad and work in all facets is encouraged and needed.  The common
denominator across the variety of approaches is solid methodology
and effective communication, so those aspects of the papers are
strong factors in our decision."
Papers are nominated for consideration, and I encourage you to do
so, but I am also here to report that amongst the reviewers our
views of what constitutes a, or the, Science of Security vary rather
a lot.  Some of us would prioritize purpose, agreeing with Charles
Darwin that "All observation must be for or against some view if
it is to be of any service"[CD].  Some of us view aspects of
methodology as paramount, especially (as hinted above) reproducibility
and the clarity of communication on which it depends.  Some of us
are ever on the lookout for what a physicist would call a unifying
field theory.  Some of us insist on the classic process of hypothesis
generation followed by designed experiments.  We vary, and I take
that to be a vote of sorts on whether cybersecurity is yet a science.
But Darwin's remark that all observation must be for or against
some view is not quite right, at least not quite right for us here.
It is not so simple as Wu's data collection, data modeling, and
decision making, either.  When you collect data and with it build
a model, your goal matters.  If your purpose in building a model
is to come to a definitive conclusion about causality, about how
nature works, then you are saying that the inputs to your model and
the coefficients that calibrate their influence within your model
are what matters.  Parsimony in the sense of Occam's Razor is your
judge, or, as Saint Exupery put it, "You know you have achieved
perfection in design, not when you have nothing more to add, but
when you have nothing more to take away."
By contrast, when your purpose in building a model is to enable
control of some process or other, then you will not mind if your
input variables are correlated or redundant -- their correlation
and their redundancy are not an issue if your goal is to direct
action rather than to explain causality.
In some circumstances you can do both, that is you can both explain
causality and enable control.  In those situations, it is your
model's ability to do prediction that both satisfies the reader
that you have captured a causal relationship and that the model's
predictions can be operationalized irrespective of any underlying
truths[MM].  A goal of understanding causality in its full elegance
leads to F=ma or E=mc^2.  A goal of control leads to econometric
models with thousands of input variables each of whose individual
contribution is neither clear nor relevant.
That all may seem harsh, but it is not meant that way at all.
Consider anomaly detection and its role in current cybersecurity
products.  Anomaly detection presumes something about distributions
of detectible events, namely that within a selected interval anything
outside some bounding box is worth investigation.  It is not concerned
with causality; it is concerned with control irrespective of
causality.  This is a coherent strategy, though with side effects.
Or consider big data and deep learning.  Even if Moore's Law remains
forever valid, there will never be enough computing hence data
driven algorithms must favor efficiency above all else, yet the
more efficient the algorithm, the less interrogatable it is,[MO]
that is to say that the more optimized the algorithm is, the harder
it is to know what the algorithm is really doing.  That was the
exact theme of a workshop held by Morgan Stanley and the Santa Fe
Institute two Octobers ago titled, "Are Optimality and Efficiency
the Enemies of Robustness and Resilience?"[SFI]
And there is a feedback loop here: The more desirable some particular
automation is judged to be, the more data it is given.  The more
data it is given, the more its data utilization efficiency matters.
The more its data utilization efficiency matters, the more its
algorithms will evolve to opaque operation.  Above some threshold
of dependence on such an algorithm in practice, there can be no
going back.  As such, preserving algorithm interrogatability despite
efficiency-seeking, self-driven evolution is the research grade
problem that is now on the table.  If science does not pick this
up, then Larry Lessig's characterization of code as law[LL] is
fulfilled.  A couple of other law professors have seized on that
very idea and suggested that price-fixing collusion amongst robots
will be harder to detect than collusion amongst people[WRC].
I am involved in a few of the many information sharing initiatives
that seem to be everywhere at once.  In one, and speaking in the
context of control, not causality, this comment was made: "Some of
your data suppliers may not want to send you data if you display
it as soon as you get it as that may be used by bad people to notice
that they have been detected or to game the detection systems.
Sure, you can obfuscate the data, but that's a lot of work to gain
a few minutes or hours of statistics."  That writer is emphasizing
that we are operationally likely to choose control over basic
research and that we have real opponents.
The point is this: if we choose control as the purpose of our efforts
then we will have to let causality become harder to see because our
models will submerge any causal relationships in a thicket of
confounding.  If, instead, we focus on causality, the very things
that we need to measure become harder to get if, for no other reason,
our sentient opponents will make it so.  As I've said many times,
I'm for measurement as decision support, i.e., I am in the control
camp, not the causality camp.  At the same time, I very much do
want to be able to ask some algorithm "Why did you do that?" and
get a meaningful answer.  Overall, having both control and
interrogatability is a difficult problem to say the least.
What I want is to predict the future.  I want it for reasons that
are no doubt emotionally clear, but I also want it because of my
own definition of security: The absence of unmitigatable surprise.
As always in cybersecurity, we are now talking tradeoffs.  One of
those is in deciding how many failures is the right number of
failures.  It can't be unbounded; that's obvious.  It can't be zero,
either, as zero quite likely means that you are overspending and,
in any case, learning from failure is especially crisp; as Francis
Bacon said "Truth emerges more readily from error than from confusion."
Defining a state of security as the absence of unmitigatable surprise
mirrors what I know to call "the availability calculus," namely
that we can get 100% availability by driving the time between
failures to infinity or by driving the time to repair to zero.  I
am searching for prediction because I want to drive to infinity the
time between failures for which I have no mitigation and drive to
zero the time to repair failures for which I do have mitigation(s).
That is where a focus on control leads you, or so I now think.
But one must still ask "Why prediction?  Why speculate?"  The late
Michael Crichton gave a magnificent essay with that exact title in
April, 2002.[MC]  Boiled down, Crichton simply said that no one
knows the future and that those who pretend to do so are self
serving, delusional, or something else equivalently uncomplimentary.
So are the people who believe what the predictors say.  He notes
how big the prediction industry really is, singling out media
especially, and he reminds us all that the track record for sweeping
predictions is pretty poor.  He coins a clinical term and I might
as well quote his text where he does so:
   Media carries with it a credibility that is totally undeserved.
   You have all experienced this in what I call the Murray Gell-Mann
   Amnesia effect.  (I refer to it by this name because I once
   discussed it with Murray Gell-Mann, and by dropping a famous
   name I imply greater importance to myself, and to the effect,
   than it would otherwise have.)
   Briefly stated, the Gell-Mann Amnesia effect is as follows.  You
   open the newspaper to an article on some subject you know well.
   In Murray's case, physics.  In mine, show business.  You read
   the article and see the journalist has absolutely no understanding
   of either the facts or the issues.  Often, the article is so
   wrong it actually presents the story backward -- reversing cause
   and effect.  I call these the "wet streets cause rain" stories.
   Paper's full of them.
   In any case, you read with exasperation or amusement the multiple
   errors in a story, and then turn the page to national or
   international affairs, and read as if the rest of the newspaper
   was somehow more accurate about Palestine than the baloney you
   just read.  You turn the page, and forget what you know.
   That is the Gell-Mann Amnesia effect. I'd point out it does not
   operate in other arenas of life.  In ordinary life, if somebody
   consistently exaggerates or lies to you, you soon discount
   everything they say.  In court, there is the legal doctrine of
   falsus in uno, falsus in omnibus, which means untruthful in one
   part, untruthful in all.  But when it comes to the media, we
   believe against evidence that it is probably worth our time to
   read other parts of the paper.  When, in fact, it almost certainly
   isn't.  The only possible explanation for our behavior is amnesia.
Everyone here knows precisely what Crichton is talking about (or
was, fourteen years ago): what is written about cybersecurity for
the general audience is often counterfactual and/or counterlogical.
Unfortunately, what is written for specific audiences like legislatures
and regulatory agencies can also be counterfactual and/or counterlogical.
And all of this finds an audience because of an actual need that I
argue is acutely important for cybersecurity -- we need to predict
the future if our tools are to intersect our problems on target and
in time.
There is a theme here -- the fast moving nature and, yes, the
unpredictability of the cybersecurity regime are such that when it
is possible to make useful predictions we are much better off, much
better able to accomplish our security plans while those plans were
still meaningful.  At the same time, and especially in cybersecurity,
no one can predict the future.  We desperately need prediction, we
know it, and it is impossible to do and increasingly so.
At the same time, I am, myself, entirely guilty of trying to do
prediction in cybersecurity.  As with today, I give speeches on
this question[GS].  I am working on a personal project right now
whose only point is prediction.  With a quant colleague, I've long
run another.  I work on the periphery of the intelligence community,
and the intelligence community is entirely about prediction --
constantly speculating on what is our actual position and what is
our actual velocity.  If your very job is security in any sense,
then you want all the prediction you can get.
Yet, at the same time, surprises happen.  If he were still with us,
Crichton would remind us that "[T]he problem with speculation is
that it piggybacks on the Gell-Mann effect of unwarranted credibility,
making the speculation look more useful than it is."  One can argue,
for example, that compliance is a predictive exercise, based on the
idea that "If you do this thing, then you can approach the future
with less to fear."  I buy that train of thought wholeheartedly,
but what if the rules to be complied with cannot keep up with the
rate of change?  If they cannot, then whatever the prediction of
outcome that compliance promises is prediction made relative to
conditions that no longer hold.  That can't be good.  Or useful.
Unpredictability is so true in cybersecurity that we have a special
name for when prediction fails: zero-day.  We accept that a genuine
0day is an attack that no one could have seen coming.  We so very
often imply that failing to handle that 0day is blameless since,
after all, it was not predicted.  Yet every time a particularly
lurid 0day shows up, I find myself thinking "Could I have predicted
that?  How?"
Bruce Schneier asked a question that is simple to state, a question
we have yet to answer, a question where getting at its answer might
change things: Are vulnerabilities sparse or dense?[BS]  If and
only if vulnerabilities are sparse does it actually make sense to
allocate the effort to find them or reward those who do or to set
up apparati for sharing information about those vulnerabilities.
If vulnerabilities are dense, then treasure should not go to finding
them but to making systems resilient to them.  This speaks directly
to security as the absence of unmitigatable surprise, does it not?
I have re-asked Schneier's question in varied settings; I have
gotten strong opinions over the full range of dense to sparse.
Smart, knowledgeable people say "too dense to measure" while other
smart, knowledgeable people say "too sparse to measure."  That's
not a trick question.  It is a steering question like no other.
In similar vein, it seems to me that the broad and general "we" are
near a fork in the road, a fork where one road is that of generating
provably defect-free code followed by long term, brutally rigorous
change control while the other road is that of moving target defense,
rapid release, DevOps, etc.  These alternatives seem both antithetical
yet promising and both are fed by real scientists making real
progress, but after having chosen one road switching to the other
at some later time seems likely to be infeasible.  This speaks again
to the "availability calculus" -- do we maximize the mean time
between failures or do we minimize the mean time to repair?  We
cannot do both nor, therefore, should we try.  I will predict that
as automation proceeds, we will find that vulnerabilities are, in
fact, dense.  The August 4th Cyber Grand Challenge contest[CGC]
speaks to this, or, as DARPA Program Manager Mike Walker put it,
"[We] achieved CGC's primary goal, which was to provide clear proof
of principle that machine-speed, scalable cyber defense is indeed
possible...  In the same way that the Wright brothers' first flight
launched a chain of events that quickly made the world a much smaller
place, we now have seen for the first time autonomy involving the
kind of reasoning that's required for cyber defense."
There is a recent paper that is very much worth your time to read:
"Global Cyberspace is Safer than You Think: Real Trends in Cybercrime"
by Eric Jardine via Chatham House last year[EJ].  Its message is
exactly that given by its title: that cyberspace is getting better
-- not getting worse, that cyberspace is getting more safe -- not
getting more dangerous.  The argument for that message is that to
think cyberspace is ever worse, ever more dangerous comes from
failing to properly normalize whatever measures of safety you've
heretofore been paying attention to.  It is only fair to quote its
front matter directly:
   Information technology (IT) security firms such as Norton Symantec
   and Kaspersky Labs publish yearly reports that generally show
   the security of cyberspace to be poor and often getting worse.
   This paper argues that the level of security in cyberspace is
   actually far better than the picture described by media accounts
   and IT security reports.  Currently, numbers on the occurrence
   of cybercrime are almost always depicted in either absolute
   (1,000 attacks per year) or as year-over-year percentage change
   terms (50 percent more attacks in 2014 than in 2013).  To get
   an accurate picture of the security of cyberspace, cybercrime
   statistics need to be expressed as a proportion of the growing
   size of the Internet (similar to the routine practice of expressing
   crime as a proportion of a population, i.e., 15 murders per 1,000
   people per year). ...  In particular, the absolute numbers tend
   to lead to one of three misrepresentations: first, the absolute
   numbers say things are getting worse, while the normalized numbers
   show that the situation is improving; second, both numbers show
   that things are improving, but the normalized numbers show that
   things are getting better at a faster rate; and third, both
   numbers say that things are getting worse, but the normalized
   numbers indicate that the situation is deteriorating more slowly
   than the absolute numbers.  Overall, global cyberspace is actually
   far safer than commonly thought.
In short, Jardine is saying that the denominator matters, i.e.,
that reporting counts of anything is poorer decision support than
reporting rates and proportions, that counts of events per unit
time will, and must, mislead.  It is incorrect to talk about how
much mayhem there is without talking about how much opportunity for
mayhem there is.
Jardine's line of critique is entirely straightforward, and cyberspace
is not the only place that such arguments about the validity of
inference are taking place.  As a prominent example, consider Stephen
Pinker's book _The Better Angels of Our Nature: Why Violence Has
Declined_[SP].  In a synopsis in the Wall Street Journal, he wrote:
   We tend to estimate the probability of an event from the ease
   with which we can recall examples, and scenes of carnage are
   more likely to be beamed into our homes and burned into our
   memories than footage of people dying of old age. There will
   always be enough violent deaths to fill the evening news, so
   people's impressions of violence will be disconnected from its
   actual likelihood.
This is, again, an argument for looking at rates and proportions
rather than counts.  But in a direct cross, Nassim Nicholas Taleb
responded with a paper, "On the Super-Additivity and Estimation
Biases of Quantile Contributions"[NT] in which his argument is that
when a distribution is fat-tailed, estimations of parameters based
on historical experience will inevitably mislead:
    When I finished writing _The Black Swan_, in 2006, I was
    confronted with ideas of "great moderation" by people who did
    not realize that the process was getting fatter and fatter tails
    (from operational and financial leverage, complexity,
    interdependence, etc.), meaning fewer but deeper departures
    from the mean.  The fact that nuclear bombs explode less often
    that regular shells does not make them safer.  Needless to say
    that with the arrival of the events of 2008, I did not have to
    explain myself too much.  Nevertheless people in economics are
    still using the methods that led to the "great moderation"
    narrative, and Bernanke, the protagonist of the theory, had his
    mandate renewed.
And to highlight his central point:
    [We are] undergoing a switch between [continuous low grade
    volatility] to ... the process moving by jumps, with less and
    less variations outside of jumps.
You may find Taleb's paper difficult, but he is speaking to our
interest in cybersecurity -- are we getting worse or are we getting
better?  Is there anything we are currently measuring that is leading
us to conclude that we are doing the right thing(s) as inferred
from measurements of what we believe to be outcomes?  Are our
inferences confounded with little understood assumptions about thin
tails (Gaussian) when we are actually in a fat tailed (power law)
situation?  Are we moving into a world where, as Taleb suggests,
we are switching from continuous low grade volatility to less
frequent but much larger jump changes in the state of the (our)
world?
So is cyberspace getting worse or getting better?  Jardine asks us
to normalize how many events did occur to the size of how many
events could have occurred, not how many did occur in an interval
of unit time.  He is correct that the possible event space is
expanding dramatically, accelerating in its expansion by all accounts.
Part of that is network extent, which I've estimated as having a
35% compound annual growth rate.  Part of that is the question of
attack surface, per se[AS].  Jardine is right that when we count
events we are misleading ourselves as to whether we are getting
better or getting worse.  But does changing the divisor alone really
make the correction we need?
There is a power law here, to be sure.  Wikipedia's concise reminder
is that "Power-laws have a well-defined mean only if the exponent
exceeds 2 and have a finite variance only when the exponent exceeds
3; most identified power laws in nature have exponents such that
the mean is well-defined but the variance is not, implying they are
capable of black swan behavior."  That, my friends, is our situation
-- cyberspace does not have a well-defined variance for what can
go wrong and hence cyberspace is unarguably capable of black swan
behavior.
Returning to Elroy Dimson's "more things can happen than will" we
must acknowledge that our rate of growth in interdependence is
absolutely making the number of things that can happen larger.
Unfortunately, complexity prevents us from counting the number of
things that can happen hence Jardine's argument that we divide the
number of things that did happen by the number of things that could
have happened is correct in spirit but would be irrelevant if our
estimate of the number of things that could have happened were to
be wrong.
Yet if the denominator is the number of things that could have
happened and we severely underestimate that, doesn't that make the
news even better?  Taleb says "no" emphatically; the fat tails of
power law distributions enlarge the variance of our estimates leading
to less frequent but more severe failures.  The best one could say
is that most days will be better and better but some will be worse
than ever.  Everything with a power law underneath has that property
(think earthquakes and whether one is overdue in California), and
cyberspace's interconnectivity and interdependence are inherently
power law phenomena.
In an article in the San Francisco Chronicle[TL], Thomas Lee recounted
how
   I found myself at a dinner in a fancy Menlo Park hotel to discuss
   cybersecurity with the executives of top Silicon Valley firms.
   [T]he mood was decidedly grim.
   "A devastating cyberattack is likely to occur in the next five
   years," said a top HP exec.  Companies are nowhere near prepared
   for it.  Neither are the feds.  There were plenty of comparisons
   to hurricanes and earthquakes.
   "A slow-moving train wreck," one executive said.
   There [is] a kind of collective cognitive dissonance in Americans'
   thinking about tech.  We'll eagerly pursue new innovations like
   the Internet of Things and electronic health records even as
   we're increasingly aware how vulnerable such technology makes
   us to terrorists and criminals.
   What struck me about the dinner, attended by executives from
   Hewlett Packard, Cloudera and PayPal, along with academics and
   investors, was the naked pessimism expressed by those in the
   room.  Nobody even tried to put a happy face on the situation.
Are those executives, academics, and investors getting the right
answer for the wrong reasons?  Are Jardine and/or Pinker getting
the wrong answer for the right reasons?  Is it a truism that when
risk cannot be estimated it will therefore be underestimated?[BH]
How do we tell if we are getting better or getting worse and how
can we explain this to citizens, regulators, and reinsurers?  Is
Taleb right that fat tailed distributions and asymmetry are where
risk lies and, which is more, that the apparent suppression of small
failures is "balanced" by yet-to-be-observed black swan excursions?
Saying it again, if the distributions of events we care about in
cybersecurity are describable not as Gaussian but rather as Power
Laws, then the best one could say is that most days will be better
and better but some will be worse than ever.  Everything with a
power law underneath has that property, and I submit that cyberspace's
interconnectivity and interdependence are inherently power law
phenomena.  The size of data breaches follows a power law, for
example.  A fat-tailed setting inherently resists prediction, but
for that very reason makes prediction ever more compelling to pursue.
So we get published predictions.  Lots of them.  Many of them hedge
their bets by phrasing their prediction as a question, but that
only invokes Betteridge's Law of Headlines ("Any headline that ends
in a question mark can be answered by the word no.").
It's a quandary.  Fast change means toolsets for protection always
trail the need unless the need can be forecast.  Fast change makes
forecasts hard if that fast change is one of adding mechanisms, not
just scale, to the equation.  We've got both scale, such as an IoT
with a 35% compound annual growth rate, and mechanism, such as
afterthought interconnection of sundry gizmos each with new interfaces.
To be deadly serious about cybersecurity requires that --EITHER--
we damp down the rate of change, slowing it enough to give prediction
operational validity --OR-- we purposely increase unpredictability
so that the opposition's targeting exercise grows too hard for them
to do.  In the former, we give up various sorts of progress.  In
the latter, we give up various sorts of freedom as it would be the
machines then in charge, not us.  Ironically, I can't even talk
about prediction with making a prediction!
--------------------------------------------
Implications: Why this matters
There is no argument whatsoever that the proliferation of devices
and information are empowering.  It is categorically true, not to
mention obvious, that technology is today far more democratically
available than it was yesterday and less than it will be tomorrow.
3D printing, the whole "maker" community, DIY biology, micro-drones,
search, home automation, constant contact with whomever you choose
to be in constant contact with, instrumentation of every stripe and
caliber, the steady migration of military technology to general
government use thence to the rich thence to the lumpen proletariat
-- these are all examples of democratizing technology.  This is
perhaps our last fundamental tradeoff before the Singularity occurs:
Do we, as a society, want the comfort and convenience of increasingly
technologic, invisible digital integration enough to pay for those
benefits with the liberties that must be given up to be protected
from the downsides of that integration?
If risk is that more things can happen than will, then what is the
ratio of things that can now happen that are good to things that
can now happen that are bad?  Is the good fraction growing faster
than the bad fraction or the other way around?  Is there a threshold
of interdependence beyond which good or bad overwhelmingly dominate?
Now that we need cybersecurity protections to the degree that we
do, to whom does the responsibility devolve?  If the worst laws are
those that are unenforceable, what would we hope our lawmakers say
about technologies that are not yet critical but soon could be?  Do
we forbid becoming critically dependent on them when it is the sheer
magnitude of their adoption that makes them critically essential?[DG]
How do we think about intelligence in this setting?  Growth in
personal power has meant that heretofore military needs, like traffic
analysis, have become common needs.  One then asks whether when
entities under attack, be they enterprises, carriers, or individuals,
garner enough provenance to engage in "strike back" should they?
As a righteous societal good, we, by law, require that persons with
disabilities not be prevented from the fullest possible participation
in our society.  Can we find the wisdom to do something equivalent
for those who cannot or, to the point, will not adopt the plethora
of technologies that are redefining what "full participation in
society" means?  Is preserving such options simultaneously the
surest way to prevent a common mode digital failure?
The need for what we have heretofore called cybersecurity is now
so varied that it is no longer a single field but many.  There are
over 800, perhaps over 1000, cybersecurity startups in some stage
of the funding game, a fair fraction of them spinouts from highly
focused university research projects.  Generalists such as myself
cannot be replaced -- there is too much for the novitiate to learn.
The core knowledge base has reached the point where new recruits
can no longer hope to someday become competent generalists, serial
specialization is the only broad option available to them.
As I say often, cybersecurity is perhaps the most difficult
intellectual occupation on the planet.  Note that I said "occupation"
rather than "profession."  Three Septembers ago, the U.S. National
Academy of Sciences concluded that cyber security should be seen
as an occupation and not a profession because the rate of change
is simply too great to consider professionalization[NAS].  Ray
Kurzweil is beyond all doubt correct;[SN] within the career lifetime
of nearly everyone in this room, algorithms will be smarter than
we are, and they will therefore be called upon to do what we cannot
-- to protect us from other algorithms, and to ask no permission
in so doing.  Do we, like Ulysses, lash ourselves to the mast or
do we, as the some would say, relax and enjoy the inevitable?  What
would we have science do?  What are the possible futures you will
tolerate?  What horses do you want not let out of the barn?  Where
do we put our intelligence budget?  US CYBERCOM's budget is $500
million, JPMorganChase, alone, is spending $600 million.  Is that
surprising or is that as it should be?
As you know, there is much focus today in the security product
market on behavioral security, on accumulating an idea of what
routine operation looks like the better to detect badness early,
but is anyone actually proposing watching 10^10 Internet-of-Things
data sources for actionable anomalies?  Presumably not, but does
that tell us of a latent need or does it tell us something else
again?  Something about redundancy or about minimizing dependence
on singleton devices?  Something about trading off the risks of
single points of failure against the risks of common mode failure?
It is likely that there are no best answers, and that all answers
will be context dependent - a threat model to rationalize attack
surface measurement for, say, medical care will be something entirely
different than for, say, shopping mall inventory control.  Any
context that actually matters will have to have an attack surface
metric (or something like it) that scales well; Qualcomm's Swarm
Lab at UC Berkeley has notably predicted 1000 radios per human by
2025[QS] while Pete Diamandis' book _Abundance_ calls for 45x10^12
networked sensors by 2035[PD].  These kinds of scale cannot be
supervised, they can only be deployed and left to free-run.  If any
of this free-running is self-modifying, the concept of measurable
attack surface is just plain over.  So, too, is the concept of
trustworthy computing, at least as presently understood.
In any case, we are past the point of no return here.  The IOT and
its scale make most of our gross measures (like attack surface,
say) into historical curiosities.  The present author has long
thought that the pinnacle goal of security engineering to be "No
Silent Failure" and with the IOT at its predicted scale, perhaps
that goal will now meet its most formidable challenge.  It may be
that for the IOT we security metricians will have to start over.
It may be that our metrics for the IOT will be less observational
and more analytic, such as "How much silent failure is tolerable?"
Surely adding 10^10 devices to the connected world increases its
complexity, and more complexity means less system predictability
which conflicts with security goals.
The general public has clearly chosen convenience over aspects of
security that seem obvious to us, but only to us, so while cybersecurity
once meant preserving confidentiality first and foremost, that is
at most quaint in a world of information sharing and sensor-based
data collection.  What increasingly matters is not data confidentiality
but data integrity.  Virtual reality means total surveillance, and
augmented reality means very nearly that, which only reinforces an
ascendancy of data integrity over data confidentiality.  Ditto
instrumented homes and vehicles.
The general public clearly expects government to provide safety as
a service.  But, strictly speaking, safety means protection against
the unintended and the unpredictable.  In the public's parlance,
safety and security merge.  There are so many examples of this it
is hard to even pick one, but to be as timely as possible, this
month's proposal to ban human driven cars from Interstate 5 between
Seattle and Tacoma is one we here should care about.  Generally
speaking, the denser the interactions between parties the greater
the restrictions on freedom required to ensure safety amongst them.
As the interaction density in the wired world continues to rise,
new limitations of freedom are inevitable.  It was the public safety
argument that forced first geolocation and then tracking of mobile
phones, and the public safety argument will soon make geocoding the
Internet into law in a similar fashion.
There is no doubt that nation states want to have better digital
boundaries than they do now.  Some states have put real effort into
this, China and North Korea continuously, Egypt and Syria intermittently,
etc.  A geocoded Internet would be one contributor to reestablishing
boundaries; the other would be naming.  Naming is the final
battleground for boundary creation.
Science is fast teaching us that everything is unique if examined
at close enough detail.  Some of it you already know; facial
recognition is possible at 500 meters, iris recognition is possible
at 50 meters, and heart-beat recognition is possible at 5 meters.[OLEA]
Your dog can identify you by smell; so, too, can an electronic dog's
nose.  Your cell phone's accelerometer is plenty sensitive enough
to identify you by gait analysis.  A photograph can be matched to
the camera from which it came as well as a bullet can be matched
to the barrel of the long gun through which it passed.  Some apartment
building owners now require that tenants provide a DNA sample of
their dog so that unscooped poop can be penalized.[BPVL]
When everything is detectably unique, decision support of many sorts
becomes possible.  Assessing nuances, such as whether you are angry,
will be embedded in automatons.  Accountability will doubtless be
extended to ever more minor behaviors.  That heartbeat recognition
technology is already slated to be part of automobiles.  Courtroom
alibis will soon be backed by cybersecurity-like evidence, noting
that because an alibi involves evidence of innocence rather than
of guilt, the privilege against self-incrimination is not implicated
and is, instead, subject to compelled disclosure.  The testimony
of spouses against each other will be unnecessary -- the testimony
of their devices will do.
As software services eclipse software products, auto update becomes
undeterable and, intentionally or not, adds weight to moving target
defense.  However, the undeterable nature of updates coupled to
ever increasing personalization will eventually make undeterable
auto update into a tool of law enforcement.
Harvard Business School Prof. Shoshana Zuboff has had much to say
on these topics since the 1980s, especially her Three Laws: [SZ]
.  Everything that can be automated will be automated
.  Everything that can be informated will be informated
.  Every digital application that can be used for surveillance and
   control will be used for surveillance and control
I think her predictions are dead-on, but the implication that this
is all outside the control of the citizen is not yet true.  It may
get to be true, but in so many words that is why I am standing here.
There are a million choices the individual person, or for that
matter the free-standing enterprise, can take and I do not mean
converting all your browsing over to Tor.  As I said earlier, when
we speak about cybersecurity policy we are no longer engaging in
some sort of parlor game.  The public's expectation of safety as a
service can hardly be overestimated.  Or ignored.
The future of cybersecurity has to now be understood as the future
of humanity.  Here's Tim O'Reilly:
   Algorithmic regulation is ... a system of governance where more
   exact data collected from citizens via their smart devices and
   computers are used for more efficiency in organizing human life
   as a collective.[TO]
This is where we are.  More things than ever could happen.  Way
more.  Only some of them will.  Almost all of them depend on what
we here know as applied cybersecurity.  We've got a lot to do.
There is never enough time.  Thank you for yours.
EOT
-------------
references:
[AS] D Geer, "Attack Surface Inflation".
geer.tinho.net/ieee.sp.geer.1107a.pdf
[BH] D Geer, "Cybersecurity as Realpolitik," 2014.
geer.tinho.net/geer.blackhat.6viii14.txt
[BS] B Schneier, "Should U.S. Hackers Fix Cybersecurity Holes or
Exploit Them?"
www.theatlantic.com/technology/archive/2014/05/should-hackers-fix-cybersecurity-holes-or-exploit-them/371197
[BPVL] BioPet Vet Lab of Knoxville.
www.biopetvetlab.com/index.php/dna-pet-id
[CD] letter to Henry Fawcett, 1863, as quoted in _Medicine in
Quotations: Views of Health and Disease Through the Ages_, Huth &
Murray, ed., 2006.
[CGC]  DARPA Cyber Grand Challenge, 4 August 2016, Las Vegas.
[DG] D Geer, "Stress Analysis," geer.tinho.net/fgm/fgm.geer.1412.pdf
[EJ] E Jardine, "Global Cyberspace is Safer than You Think: Real
Trends in Cybercrime", 2015.
www.cigionline.org/sites/default/files/no16_web_0.pdf
[GS] D Geer, "What Does the Future Hold for Cyber Security?", S&S,
2015. geer.tinho.net/geer.suitsandspooks.19vi15.txt
[JW] www2.isye.gatech.edu/~jeffwu/presentations/datascience.pdf as
drawn from en.wikipedia.org/wiki/Data_science
[LL] Lawrence Lessig, _Code: And Other Laws of Cyberspace_, Basic
Books, 2000.
[MC] M Crichton, "Why Speculate?", 2002, reproduced at
geer.tinho.net/crichton.why.speculate.txt
[MM] "You see, there is only one constant, one universal.  It is
the only real truth; causality." -- Merovingian/Matrix
[MO] Michael Osborne, Oxford University, personal communication,
2015.
[NAS] "Professionalizing the Nation's Cyber Workforce?"
www.nap.edu/openbook.php?record_id=18446
[NSF] D Geer, "T.S. Kuhn Revisited", 2015 geer.tinho.net/geer.nsf.6i15.txt
[NT] N Taleb, "On the Super-Additivity and Estimation Biases of
Quantile Contributions", www.fooledbyrandomness.com/longpeace.pdf
[OLEA] OS-3001, www.oleasys.com
[PB] P Bernstein, _Against the Gods_, Wiley, 1996, plus
www.mckinsey.com/insights/risk_management/peter_l_bernstein_on_risk
[PD] P Diamandis, _Abundance_, Free Press, 2012.
[QS] via "Roadmap for the Trillion Sensor Universe", Qualcomm Swarm Lab
[SFI] "Optimality vs. Fragility: Are Optimality and Efficiency the
Enemies of Robustness and Resilience?", 2014.
www.santafe.edu/gevent/detail/business-network/1665
[SG] S Gillon, _That's Not What We Meant to Do_, Norton, 2000.
[SN] R Kurzweil, _The Singularity Is Near_, Penguin, 2005.
[SP] S Pinker, _The Better Angels of Our Nature_, Viking, 2011.
[SOS] NSA, Science of Security
www.nsa.gov/public_info/press_room/2014/Best_Scientific_Cybersecurity_Paper_Competition.shtml
[SZ] S Zuboff, Harvard Business School
. Everything that can be automated will be automated.
. Everything that can be informated will be informated.
. Every digital application that can be used for surveillance and
.    control will be used for surveillance and control.
[TL] T Lee, "Forget Target, Ashley Madison Hacks, a Bigger Threat
Looms," 2015.
www.sfchronicle.com/business/article/Forget-Target-Ashley-Madison-hacks-a-bigger-6395645.php
[TO] T O'Reilly, "Open Data and Algorithmic Regulation," in _Beyond
Transparency_, Code for America Press, 2013.
beyondtransparency.org/chapters/part-5/open-data-and-algorithmic-regulation
[TSK] TS Kuhn, _The Structure of Scientific Revolutions_, U Chicago,
1962.
[WRC] "When Robots Collude" (algorithms can learn to do so), 2015.
uk.businessinsider.com/robots-colluding-to-manipulate-markets-2015-4
This and other material on file at http://geer.tinho.net/pubs
原文链接:http://geer.tinho.net/geer.uncc.5x16.txt