A simple proposal to fix the Internet

I've developed a really simple theory about why the Internet is so messed up, and how we can fix it.

The Internet is fundamentally broken because all the power is held by the services you use.  And that's working out about how you'd expect, with this asymmetric control leading to all kinds of nasty problems -- for example, privacy abuse, mass surveillance, and censorship.

But what's the source of that power imbalance?  It's your identity.  Specifically, it's the fact that the Internet is designed so that user identity is owned by the service provider, not the user. And it's what drives the bizarre condition where you need to give your stuff away to strangers for it to be useful to you.

My theory is that's all you need to fix.  Below I will propose a free and open cloud service that lets you manage your own identity, and keep all your stuff private.  That fixes all the problems above instantly.  More importantly, you'll discover that your stuff is waaaay more valuable to you when you don't give it away.

Imagine this simple cloud widget:

This service empowers you to manage your OWN identity on the Internet, instead of letting strangers do it for you. Let's call it a cloudspace. It's completely free, and works exactly like all the cloud services you use today, except it's all in one simplified place.

The key difference is that, when you sign up, instead of adding your stuff to some big shared database, the service creates a private database just for you.  That's your cloudspace.  It's where you stash all the digital stuff you now scatter all over the place. And since the cloudspace is also your communication app, it saves data from all your interactions going forward.

Your cloudspace is completely private; it's encrypted with a key only you possess, so even your cloud service provider can't see anything inside.  You own this database -- both legally and administratively -- so you can move it anywhere you want, including to competing services.  And since everyone's cloudspace is encrypted separately, there's no central point for anyone to spy on, censor, or attack.

Today most of your digital stuff is generated by interactions.  Therefore your cloudspace acts as your personal interface to the Internet -- to other cloudspace users, as well as the Web. It's designed so you can explicitly declare your identity for any interaction; you can even have multiple identities, or remain completely anonymous.

Cloudspaces can "friend" each other, permitting people to self-assemble into any (private or public) networks they choose. All the social/communication modalities you use today are supported, only better, because (among other things) all interactions between cloudspace users are encrypted. And organizations and businesses can also participate, so your cloudspace enables you to manage all those relationships in one place.

Your cloudspace is radically programmable, which is what enables it to perform any social or collaborative function, including future innovations. Anyone can write a Facebook-like or Twitter-like app, for example.  What's different is that apps can only read/write to your cloudspace, not take your data away to some place you don't control. Rich open APIs, and the breadth and depth of your data, will drive amazing solutions, all for your sole benefit as data owner.

The cloudspace software uses existing cloud technology.  Its core design is open source and open standard, so no one can ever own or control it. Economically, it removes all those perverse incentives that have screwed up the Internet, all while keeping it free for you and me.

But most importantly, it restores privacy as the default condition for human identity on the Internet.

----------------------------------------------

The narrative above is derived from a simple insight I had in late 2013, when I was thinking about all the bad things we suffer on the Internet today.  Instead of viewing these problems separately, I scanned for a common weak link, and it kind of jumped right out. The age-old identity weakness -- represented by the humble login/password paradigm -- drives everything from privacy abuse, surveillance, and censorship, to cybercrime, to everyday pains like password management and data backups.

In May I wrote this idea up and published it here on the blog, where nobody would see it.  I’ve had opportunity to refine and update that piece since.  That's the definitive document with the white paper-y treatment, including the technical specifics (spoiler alert: it's a standard JSON database with an API wrapper).

Below I will summarize the main points of the theory; this should enable you to start seeing the world as it would be with a user-centric identity model in place.  If you are like me, you will have a moment every day that proves the value in fixing identity.

-----------------------------

Here’s the opportunity, in a nutshell:

Instead of accepting a state where our identities are created and managed by others, we simply need to claim control of our own.

As I noted above, the root problem is the login/password paradigm.  It’s a model that dates to the earliest mainframes.  It’s been that way for so long that we simply don’t question it: our presumption is that the host is responsible for the creation and management of its users and identities.

But that model means that, once we create an account and start putting our data somewhere, it becomes impossible to move or in many cases even retrieve our data.  And if our friends have accounts with the same services, that raises the exit cost substantially too. And in cruel irony, this multi-account complexity is so painful, it's sparked the emergence of the "Login with Facebook/Google" buttons... quite possibly the worst idea ever, since it merely grants more data to the corporations that represent the biggest problems already.

That's why these services can continuously turn the screw on us in their Terms of Service (ToS) updates.  Facebook, for example, now tracks you across the Web and by your physical location.

But does it have to be this way?  Actually, not at all, especially in the era of the API Economy, Semantic Web, and Internet of Things.  With Cloud Composable Apps, that's not even a very hard technical problem anymore.

The cloudspace represents a new class of Internet “endpoint,” one that exists solely to represent the interests of the end user.  (You and me.)  The cloudspace is designed with two major concepts in mind:

1. It’s a single place to store your “stuff” -- everything from files and videos, to your social interactions, to the data generated by your Fitbits and Nests and the coming Internet of Things.  It's completely private, except for the things you explicitly and precisely share.
2. It’s your entry point to the Internet, where all interactions with others, and with companies and services, and any digital artifact, can be done in context of your own definition of who you are, not someone else’s (partial/corrupt) image of you.

The changes from adopting this model are instantaneous and dramatic:

• Nobody can see your stuff anymore except you, and even your interactions with others can only be seen by participants.  That kills privacy abuse, surveillance, and censorship in one shot.
• You no longer rely on third parties to facilitate interactions; these happen directly between users, with complete privacy. The result is a new “atomic Internet,” where you're logically equal to every user, and to the services you choose to use.
• With a framework in place that supports authenticated, secure interaction between any two parties, you gain abilities to dramatically improve everything from commerce to content rights to legal arrangements (e.g., no more “contracts of adhesion”).
• Because only a free and open source approach can deliver this framework, it insures against anyone having the ability to ever again assert control over your identity.

We've all heard the experts say "there's no technical solution to these problems."  I call BS.  As a product guy, I know you've just got to find and exploit your natural leverage.  In this case the end user has it, as the creator of the data.  The cloudspace simply asserts this leverage, and puts it to work for the benefit of the end user.

I know that a lot of people -- I'd say most people I know -- would love a solution to the Internet's problems of privacy abuse, surveillance, and censorship.  I think I've envisioned one that works, and actually does a lot more than that.

But as I also stated in my earlier post, my sole objective is to initiate this conversation and drive people to recognize the root problem of digital identity.  If we can fix that, I am convinced we can literally fix the Internet. I'm happy to hear any thoughts that help us toward that outcome.

Identity 2.0

Is it just me, or has the Internet been turning into a really awful place?  The privacy abuse, surveillance, and censorship alone make it painful to contemplate the future, or even envision solutions.  We're like the proverbial boiling frogs, and the heat's turning up so high we can't help notice it, but yet we still don't jump out of the pot.

I was thinking about how ridiculous and depressing this was, when I suddenly realized that maybe we're not looking at the right problem.  The real root problem -- for all of it -- is identity.  And that's actually much easier to fix.  And we can do that ourselves.

See if this makes sense.

In the digital world, we’ve been split into a zillion shards of data, which are stored and traded by people we don’t know, and who continuously harvest it in pursuit of profit.  Our personal data has literally become the raw material for the bad behaviors we see.  We've lost control of our identities.

But maybe the problem isn’t that anyone is taking away our identities.  Maybe the problem is that we’re willingly giving them away... to strangers.  And when you put it that way, it's obvious that cannot ever end well.

So... What if all our stuff just remained private?  What if it were visible only to ourselves and those we choose to share it with?  What if all our interactions took place directly between us and our network, without the need for any third party services at all? 

Well, we could have social networks without Facebook, tweeting without Twitter, photo sharing without Instagram, email without Gmail, and IMing without, well, whatever multiple networks we all use today.  There’d be no need for YouTube or Tumblr or Pinterest or WhatsApp or SnapChat or Dropbox or any of it.

What if we could simply bypass these services altogether, and do all that creating and sharing privately amongst ourselves?  What if it was just… us?

I’ll tell you what would happen.  We’d take back our digital identities.  With this simple flip in perspective, we'd gain active command and control of our digital lives.  At the same time, we'd end all of the problems that result from giving our most personal and valuable stuff away to strangers.

We’d each retain ownership of everything we create, and be in precise command of what is shared, with whom, and under what terms.  Since we’d possess all our data (both things we create and things that are shared with us), we’d gain the ability to view and search and present everything in ways that simply don’t exist today.  And as an incidental outcome, we’d collectively create a whole new identity framework, one that would drive major innovation anywhere security and privacy are important.  Which is everywhere.

In this post I will propose a model that, I believe, can achieve this vision.  Its core component is a new atomic element for the Internet, the cloudspace.

The cloudspace interoperates seamlessly with today’s Internet, while adding a missing layer of personal privacy.  It supports every feature of every cloud service or social network, yet improves upon all of them in fundamental ways.  And like the World Wide Web, it’s free (free as in beer, and free as in open), so there's no owner, and no one can ever extract a tariff from the people who rely on it.  At the same time, it provides vast opportunities for innovation and even monetization -- just not in any way that involves seeing our private stuff.

And because it must be, Digital Identity 2.0 is a model that’s completely opt-in, at the beating heart level.  Anyone can join, and nobody can stop anyone else from joining.  The benefits start the moment there are two participants, but grow exponentially, in proper network effect fashion, with each person who adopts it.

To understand the proposal, first you need to understand how badly digital identity is screwed up today.

Identity 1.0

In a nutshell: the world has completely botched the implementation of identity in the digital world.  We're still using the same login/password model, unique to each service, that predates the Internet.  Is that really an accident?  Think how far we've come in so many other areas.

This seemingly prosaic annoyance is actually the root cause of many of our biggest problems.  Because we accept a 1960s-era identity model, control has been effectively surrendered to the people who provide these Internet services -- even though we instinctively know that they are more dependent on us than we are on them.

Because it seems out of our individual control, we accept all these awful problems as the price of creating and sharing content.  Every day we suppress this resentment as we spray many shards of data across multiple apps and services, where we explicitly give strangers control over our content, and allow them to monitor our actions.  (Prime example: Facebook has been able to track us across the Internet since its last terms of use update; in 2015 they'll be able to track us physically 24/7 via GPS.) 

When you consider all this, it becomes obvious that the only real solution is to stop letting those apps and services see what we do in the first place.  But clearly that won’t happen if it means foregoing all the things that these services enable us to do -- all the posting and tweeting and sharing and IMing and emailing.

Here's the dirty little secret.  None of that stuff we sell our souls for is magical, or even remotely hard, from a technical point of view.  Virtually all of it is defined by open standards and/or established conventions.  The only leverage is our need to hang where our friends are hanging, and our collective illusion that we need Facebook (or whomever) to do that.

Drop that illusion and things change in an instant.  For the people violating our privacy, spying on us, or trying to control what we see (hi Zuck), the nightmare scenario is a simple one.  If we grab control of our own identities, we will starve the Internet of the very content that it needs in order to abuse us.

And let me be crystal clear about this.  Once a cloudspace framework is in place, Facebook is obsolete.  Nobody needs it any more, and I suspect most people will be more than happy to escape its clutches.  In fact, obsolete is ANY service that relies on user data to profit: Gmail, Twitter, LinkedIn, Dropbox, Instagram, SnapChat, YouTube, Tumblr, Pinterest, Uber, etc., etc., etc.

Because, when we can do all this stuff ourselves, privately and securely, what's the value of those services going forward?  If they adapt fast enough, there may be a way to retain some partial value in directory or orchestration services, but good on them if any can make it worth our money.  On the other hand, any profit model built solely on seeing our private stuff is well and truly borked.  And that's a good thing.

Identity 2.0

To visualize the changes described above, consider this simple diagram.  (Click to embiggen.)

Today is Digital Identity 1.0.  All interactions take place between us and some cloud service, which then completes a corresponding transaction with our intended party.  That’s what lets these services see what we do -- they insert themselves as middlemen.  Then, through contractual terms of use, we are rendered subservient.  And we must remain that way to keep using those services.

With Digital Identity 2.0, everything transacts directly between the parties, with no middleman required.  The difference is the green circles, which represent each user’s cloudspace, and automatically handles all interactions without compromising privacy or security.

Technology Architecture

The model requires three primary components; all three are net new solutions, but are built on existing technology -- some of it only recently available with the emergence of the "API Economy."

The main component is the cloudspace, which is simply a digital identity database -- a private “lockbox” for all the content you create, and where you manage sharing when you choose to do so.  It's a personal data vault: it's like your Documents folder, plus your social interactions, plus everything you generate in the future.

Once the cloudspace is in place, two new categories will complete the picture: cloud hosting to handle the database interactions; and apps to manipulate and present the data.

1. Cloudspace
The primary component is a standard JSON database -- a modern, cloud-aware database, to be sure; and we’ll take full advantage of its capabilities.  But it’s just a data bucket, like your device hard disk or Dropbox or Google Drive.  And since it’s just a single computer file, it’s compatible with any technology or platform.

Each user instantiates his or her own cloudspace; the signup requires only an email address.  What happens behind the scenes is what's different. 

The service creates a private data store for each user, encrypted so only the user can see.  Everything inside is manageable, via any tool written to the APIs.  This becomes your personal filespace, for all the things you currently store locally or on a network drive -- files, photos, videos, music, etc.  It also houses all the social or collaborative content you create -- your posts, tweets, IMs, email, etc. -- and orchestrates all the stuff that is shared by other users.

Perhaps most importantly, it's the destination for the coming explosion in personal data that will be generated by the "Internet of Things" -- all the GPS, Fitbit, Nest, home automation tools, etc. that will proliferate in the next few years.  That stuff is now scattered everywhere and it's growing worse; shouldn't it all be someplace only you can see?

Unlike all the drives and backups we need today, this single data repository can grow with you over your complete lifetime, since it’s cloud hosted and managed.  And because the cloudspace is standardized and self-contained, moving between cloud hosting services is easy and fully automated.

Through your content and interactions, your cloudspace forms the authoritative digital representation of your identity.  It grows and changes with you -- just like a real identity.  It's a single place to manage your digital identity; you're in complete control.

The database itself contains no application logic; it really is just a container.  But it has some quite useful data features, including the ability to sync efficiently/differentially across multiple copies, and a rich API set to expose its contents securely to other databases and applications.  It will have a radically extensible schema to support virtually any data type, now and into the future.  And it will have a ridiculously long private key that will prevent its encryption from getting cracked by anything short of a future quantum computer -- yet still be upgradable to keep up with such advances over time. 

There are a couple other critical elements to the design of the identity database.  First there’s a certificate that identifies you as the owner of the database, to other users and to applications.  It requires no third party certificate authority, because like your offline identity, its validity is proven over time based on your activities and relationships (e.g., a cloudspace that pays all your bills is pretty sure to be you).  It works because it always represents you -- just as a real-life identity does.

The other critical element is your contact list, which in this model becomes your social network.  Much like getting “friended” on Facebook, someone can request to be added to your contact list.  Since this is an automated process between the two users’ cloudspaces, the contact can be stored (and continuously synched) complete with metadata describing membership in public/private groups and other unique constructs, as well as preferences concerning communication and sharing.  And since your social network is now in a place where it’s completely under your control, it’s easy to fine-tune your personal groups for easy sharing, in a way that works the same across the different interaction modes. Because of these factors, the cloudspace model will peg the EFF’s Secure Messaging Scorecard.

The database itself must be open source, perhaps derived from Apache CouchDB or another mature player in that space.  Open source code is critical for this component, to eliminate the possibility of “back doors” that can hide within closed source software, and to assure that all APIs are known.  Its open source nature keeps the critical storage component of the cloudspace from ever falling under the control of anyone who can extract tariffs.

With this infrastructure in place, other opportunities to improve digital interactions appear.  For example, if you are like me, today your electric bill appears as an email notification, and you go to the company’s site to pay.  Then you get an email confirmation.  The transaction takes place fully on the company’s site, and they retain all the information, not you.  Sure you can always go and review your records there, or save the emails, but you'd need to do the same for your car payments, mobile phone, and for all other specific customer relationships you’ve accreted over the years.

With your cloudspace, the electric company could simply share the bill with you (companies can have identity managers too), and you could review and push payment in your cloudspace's UI -- theoretically the vendor wouldn't even need your payment account info.  Then everyone has a verified and complete record.  On your side, you could view all your bill payments in one place, or even integrate with financial apps automatically.

2. Cloudspace Hosting Services
For your cloudspace to be useful, it must communicate with other cloudspaces, and that can only take place in the cloud.

It will require the development of a new service type, but one that’s little different from Dropbox or Google Drive.  The big change is these cloud services mimic a file system, but your cloudspace does that for you.

The host only knows you have one file, and must simply support the API orchestrations.  This is how cloud is changing technology, as today leading cloud services handle billions of API calls every day.  It’s all about making the authenticated connections, at speed and at scale.

Like the cloudspace database, there will be an open source cloud hosting app, probably based on OpenStack and Docker, so anyone could offer it.   Service options could range from fee-based to ad-supported to free, but will compete on the speed and reliability of their API processing.  A power user might happily pay a modest price for high performance.  A casual user might accept more latency, or (non-profiling) ads, for a free service, while still gaining all the identity benefits.

In some cases the APIs will be used to actually deliver data (e.g., email and IM), where in others they might use links or pointers (e.g., video and file sharing).  The critical part is that all of these activities take place within the context of each user’s authenticated identity and assigned permissions, and are encrypted end to end.

Imagine one big change: with some simple directory/aggregation services (an opportunity vector for ecosystem players), Youtube would be obsolete. People would just post their videos to their cloudspaces and specify the sharing as public, and the cloud service does the rest.  This would be especially attractive to bands or other organizations, who could decide how their content is presented (e.g., with/without ads), and establish direct relationships with their viewers.

That’s the primary role of the cloud hosting services.  Note that most services will also play in the app component space, discussed next, with device and/or browser-based capabilities for both content management and administration.  Notably, this will likely become your browser home page -- a highly customizable aggregation of everything important to you personally.

3. Apps
Equipped with this private digital space, you will need apps to manage all the posting and sharing and IMing and emailing -- equivalents to tools and services you use now on the public Internet.  Since the cloudspace really only has storage capability, the operational capability must be provided separately.  As you will see, this is a strength of the model, as it allows for universal platform support, and provides dramatic differentiation possibilities.  What now requires a complete service, with redundant database/hosting/sharing capabilities, instead only requires an app -- because the hard part is already done.  It’s especially appropriate for the mobile device space, where rich native client apps are in high demand.

It’s through apps (including the hosting service app discussed above) that you will interact with your cloudspace.  Apps may also range from free to ad-supported to commercial.  Like the cloud hosting component, if someone delivers value that people are willing to pay for, there’s a well-understood business model to use.

When you log into your cloudspace, your app(s) will interact with the content according to specific permissions you grant.  Apps provide the user interface, and abstract the functionality inherent in the cloudspace APIs.  In this way, you could choose an app based on lots of different factors and preferences.  For example, you might want an app that comprehensively manages your identity across different modalities (e.g., social, email, IM).  Or you might prefer targeted apps for a particular function (e.g., editors for files you create and store in your database).  Or you might want different apps doing the same things across mobile and desktop devices.  Since they’re all using the same data source, it’s completely flexible.

Similarly, the cloudspace also contains APIs and other constructs that are especially useful in its role as an identity management tool.  For example, it contains the aforementioned certificate services for authentication and encryption.  It also has a live friend/contact list, to richly manage relationships and groups.  And it has a facility for managing multiple aliases, so we may present as different users to the Internet (or as anonymous), yet see everything in one view on our side.

Other innovation opportunities open up simply because the data is in one accessible place.  For example, instead of being stuck with whatever sorting algorithm a service wants to force on its users  (“Top Stories,” anyone?), you could filter and tweak your social feed based on an app’s innovations in this area (please a “mute user” button!).  Or you might use an app that combines all content into one feed, creating a truly “universal inbox.”

Because all the components are cloud-aware, apps have a lot of flexibility.  For example, an email app may request a local copy of all email content, while a social network app might prefer to leave its data in the cloud and access it remotely, allowing the cloud service to pull all the data together for presentation.  That option exists in the cloudspace's APIs, which can create and sync subset copies of content based on the data requested.

The cloudspace also delivers state-of-the-art security, supporting multiple levels of permissions.  The ability to read or post something, in a specific app, could be controlled by a simple password (or fingerprint).  Escalated rights, perhaps with 2- or even 3-factor authentication, might be required for full data views, configuration changes, or content deletions.  These capabilities are also built into the cloudspace, not the apps you use, but would create great flexibility in app design.

Finally, since the apps can surface any data stored in the cloudspace, you gain capabilities you seldom see now on any public service, let alone all the services in one view.  It's the personal panopticon.  You will finally be able to find that old post that you made and want to comment on again.  And the ability to do personal analytics across your complete data set, privately, can deliver personal value without the profiling you get on the public Internet.

You could even apply digital rights management (DRM) to the content you share, something that's loathsome in the way the DMCA defines it, but extremely valuable when everyone's an equal.  You could delete something in your cloudspace with the assurance that it will be deleted everywhere, or you can prevent forwarding/resharing, etc.

Conclusion

If this post contains one insight, it's this: the only way to keep our identities from being abused is to keep our identities private.  But to do that, we need to change the fundamental nature of digital identity.  Fortunately, that may be relatively easy to do -- at least when compared to individually addressing all the identity-based problems we suffer today.

With the cloudspace, I have attempted to describe one possible solution to the digital identity problem -- one that, if it actually works, will truly restore power to the user, and solve most of our privacy and surveillance problems.  That's huge by itself.  But it will also greatly mitigate other issues like censorship (everything is encrypted), spam (every user is authenticated), passwords (you only need one), platform inconsistencies (native apps can all use the database), and system crashes (all your data is in the cloud, backed up).

I wrote this because, like many people, I am extremely troubled -- no, offended -- by these basic, but seemingly intractable Internet problems.  I just feel that there’s got to be a better way.  To me, the technical challenges seem solvable -- although like all product managers I have at times been guilty of underestimating development complexities.

I also understand that there are powerful corporate and government interests who don’t want all your stuff to “go dark” to them.  And I don’t underestimate just how hard they would fight.  Just the disruption to existing advertising and monetization frameworks would be huge.  But I also think that, with this approach or any other that enables direct, private, encrypted interactions, there’s not much anyone could do to stop it. 

And really, if Facebook or any of the other disintermediated companies were to be smart about it, they’d leverage existing skills and insights to profit from the app and/or cloud hosting opportunities, conceding that their business model must pivot away from profiting from our personal information.  But if I was to bet, they'd probably be outmanoeuvred by someone who does it better, and/or gets there faster.

So that's my simple proposal to fix the Internet.  I do hate to appear un-humble, but in this case, yeah, the ambition is that big.

I published this as a blog post for peer review -- the foundation of science, even among crank bloggers.  But even if this approach is proved unworkable, I’d like to hear what people think -- either publicly here, or privately via links at the top.  Most of all, I want to contribute to the discussion that we need to have about how identity should work in the digital era.

Arthur Fontaine
May 2014

General Systems Theory

Introduction

General Systems Theory (GST) has been called "the skeleton of science." That's pretty apt, because GST illuminates the scaffolding upon which the universe is built. 

In its simplest formulation, GST states that everything is a system. Once you realize that, you begin to see behaviors and characteristics that are common to everything in the universe. These simple concepts ease the process of understanding anything -- from the simplest physical structure to the most complex conceptual construct.

GST was developed by Austrian biologist Ludwig von Bertalanffy starting in the 1930s, culminating with the 1969 publication of his "General System Theory: Foundations, Development, Applications." In his lifetime, Von Bertalanffy was probably better known for his mathematical model of an organism's growth, but GST was what he considered his legacy.

At the time, GST was hailed as a major scientific framework, considered one of the best unifying theories since Darwin's Theory of Natural Selection (with which of course it is 100% consistent). Anthropologist Margaret Mead was among system theory's proponents, applying its concepts to the development and disintegration of societies and civilizations. In fact, over the past 50 years systems theory concepts have become core elements of many specialized disciplines – most notably the psychic and social sciences, where systems theory is applied across a range of pathologies. 

But General Systems Theory is the meta version of systems theory, "zooming out" to encompass the entire universe and everything in it. So really, when you're talking about something that seeks to be the unifying theory of everything... well, the fact that you've probably never heard of GST says it's fallen short of its ambitions.

My awareness can be traced to a philosophy course I took in 1978 with one of GST's adherents, T. Downing Bowler, Ph.D., at the now-defunct Bradford College. It was an 8 AM class so my recall of the details was never that great, and I only got a C.  But I always remembered the concepts as being pretty cool to my 18-year-old self. A few years ago I tracked down a copy Dr. Bowler's 1981 book, "General Systems Thinking: Its Scope and Applicability," the draft of which had been the source of our (mimeographed) course content.

I found it as interesting as I remembered, and after reading the book I discovered that GST is just, well, a damn handy thing to have in your life toolkit. If you're puzzling through a problem, running it through the GST engine will almost always make things clearer. Having lived with this for a while, I felt compelled to share it.

I tried to boil this document down to as few words possible; in fact, my small contribution to the discipline may be in the application of the "80/20 rule." If you understand the 20% represented in the following core concepts, I believe that you'll get 80% of GST's practical benefit. Of course, if you find it interesting, you can go as deep as you like in the literature, discovering many concepts I will intentionally omit here.

We'll first discuss the core GST concepts, (mostly) without the use of examples; in writing this I discovered that you really have to "load the whole thing into memory" before you can start effectively using it. After outlining the framework, we'll go ahead and explore a few examples to illustrate how GST applies to our universe of disparate things. Then you're equipped to start using it on your own stuff.

What is GST?

GST works by flipping a simple mental switch. Instead of focusing on the vast and wonderful variety of the universe, GST restricts its focus to a few mechanical elements that are common to everything. Then there's no longer an infinite number of differences, but a small set of simple concepts, which are realized by our world in an infinite number of ways.

Consider that the default unit of our world is a "thing." The human mind has evolved to organize our perceptions around things, and for good reason: our ability to distinguish one entity from another is what enables us to understand and interact with our world. Thus we evaluate our entire existence through the infinite differences that distinguish our universe of things from each other.

GST is also based on things. But von Bertalanffy's insight was that there's a system to even being a thing. And that part is 100% consistent, no matter how many differences may otherwise exist between specific things. GST works by exploiting the fact that systems reliably behave as systems will.

Carving away the complexities can be wonderfully clarifying. GST is very good at resolving differences between expected and actual behavior (troubleshooting). As importantly, GST can often predict how things will behave or change in the future. 

Here are the core components of GST:

• System -- This is the basic unit of GST, and what makes it so broadly applicable. In GST, everything is a system, and there are no exceptions. Since the universe is comprised of things, it is comprised of systems; this equivalence dictates that everything you can perceive is subject to the logic and rules of GST.
• Equilibration -- This is the key process in GST, because it incorporates the concepts of time and change. Unlike equilibrium, which is a (theoretical) state, equilibration is the perpetual process of seeking equilibrium. Equilibration is necessary because of continuous exposure to impacts (stressors) originating from other systems. The primary types of equilibration are accommodation and adaptation. The effects of equilibration are manifested as change in the system over time.
• Relations -- The primary dynamic in GST is how systems relate to other systems. The sum total of all a system's relations constitutes its environment. The reasons specific systems relate, the specific manners in which the systems are related, and the strength of these relations, are all central to the nature of each system, and how it behaves and evolves. A system's relations can be very complex and involve multiple elements of competition, cooperation, and control.

And here are some of the core concepts:

• Boundaries -- The ability to distinguish where one system ends and another begins is dependent upon boundaries, defined by an observer's perception as something unique and independent. System boundaries exist because of constrained variety -- that is, faced with the limitless system relations available in the universe, each specific system exercises constraints in its relations that, in total, define its essence.
• Stressors -- A system equilibrates in response to stressors presented by other systems. While there's an element of strict stimulus/response in GST, stressors are quite rich and varied – what’s important is that they apply pressure in ways that challenge system equilibrium, and they generate equilibration responses.
• Polarities -- Implicit in the idea of stressors is the concept of polarity. As the stressor affects the system, a polarity is created between the system's (theoretical) equilibrium and the stress being applied between the two. That's not to say that all polarities exhibit the same range, but the poles always represent 100% of the difference, and define the range of possible equilibrations.
• Hierarchy and Matrices -- The universe itself is a system, and everything it contains is a participant in that system. This organization means that every system is also both a supersystem to its component systems, and a subsystem component to some larger system(s). The principle of layered organization addresses the way that systems are naturally built upon simpler systems. In highly layered or matrixed systems, it's not uncommon for two systems to relate in very specific ways, but otherwise have very little interdependency.
• Autonomy and Dominance -- Systems are autonomous by nature. However, a system may be controlled by another system, and take on the role of subsystem. As there are typically many subsystems in every system, this introduces the dynamics of subsystem competition and cooperation, as well as the command of the controlling system. These relationships can be strong influencers of system behavior across multiple levels.
• Bonds -- Relations are comprised of bonds; in fact a core GST variable is the type and strength of bonds between two systems. Bonds are subject to selectivity and discrimination. As discussed, no system could accommodate every possible relation. What happens is that systems tend to develop relations with beneficial impact, and avoid relations with destructive impact.
• System State -- There are two types of system state: stable state and steady state. A stable state system is consistent at the molecular level, and will change little in the absence of extraordinary stresses. A steady state system is typical of a living entity, characterized by intense, continuous equilibration, and a finite, relatively predictable term of existence.
• Mapping -- Mapping is how systems integrate through the exchange of information. Mapping can be ad hoc but is commonly driven by existing models. This is one topic where an example is edifying, so I’ll use it: DNA is the mapping mechanism for living organisms on Earth. The capability to map is typically a higher-order characteristic, relying on pattern recognition and learning as important processes.
• Crisis and Transformation -- A system may experience a stressor for which it has no effective equilibration option available. This system will experience a crisis and must transform -- either disintegrate or synthesize. Disintegration means that a system ceases to be, and relinquishes claim over any component systems and participation in any supersystems. Synthesis occurs when two systems combine into something that is recognizably different than either was before; implicit to the concept of synthesis is novelty, or new characteristics that fundamentally change the nature of the systems from which it was derived.

 

Practical Application

Some of those concepts might seem a bit academic. But the beauty of GST is that it’s just logic. You’re applying GST concepts every day, without realizing it. Think about the elements that go into judgment, analysis, and troubleshooting. GST simply identifies and describes the tooling, and pulls it all together into a formal framework.

The first principle in applying GST is to make sure you’re examining the right system. We’ve all made the mistake of assuming one thing to be the problem, only to discover it was actually something else. With GST, often you will realize you are dealing with stressors and equilibrations involving different systems, or additional systems, than the one you started with.

Next you focus on the equilibrations. In GST, the equilibrations are the symptoms of the stressors causing them, so matching equilibrations and stressors is key. Commonly you will find that there are multiple equilibrations and stressors in play. In this case, polarity can be a useful tool – since paired equilibrations and stressors are at polar opposites of their specific spectrum. Importantly, this exercise will often alert you to future possible equilibrations, should particular stressors increase or change.

Lastly, much can be gained by examining the relations between various systems.

• What are the dynamics of control, competition, and cooperation between systems? In one common scenario, two systems may be acting in cooperation for the benefit of a controlling supersystem, while simultaneously being in competition for resources within that system. Or a system may try to attain dominance itself.
• What are the bonds between systems? Are they appropriate or complete? If bonds persist after they are no longer useful, this can cause unhelpful equilibrations. If bonds are quite strong, that can cause individual systems or even supersystems to fail unexpectedly when relatively small stressors are experienced.
• What are the system biases in play? Remember, bonds are subject to selectivity and discrimination. It’s often just as clarifying to examine places where no bonds exist – and why – as to study the bonds you see.

Examples
Let’s walk through a few broad examples to see GST in action. This high level treatment leaves opportunity to drill much deeper using GST principles; of course you can also choose to start exercising your new GST skills on your own life and environment.

• Rocks -- You might say, "That's no system, it just sits there. It’s a rock." But it really does follow all the GST rules. A rock is a stable system, so it maintains a high degree of overall equilibration. But as with all systems, its stability is directly related to the stressors to which it is subjected. While a child sitting on the rock may elicit no equilibration response, a stick of dynamite certainly could, and quite possibly cause disintegration. And you may not see that rock equilibrate much in your lifetime, but over geologic time, equilibration will show clear effect.
• People and Societies -- If you're like me, it feels dubious -- and a little insulting -- to posit that any scientific system could address the infinite variability of the human psyche. Well, it turns out that GST is especially useful when analyzing psychic and social systems, which are absolutely subject to the process of equilibration.
       Psychologists derive insight from the impacts of various external and internal stressors on a psyche, e.g., social/family pressures or mental illness. Sociologists can use GST to frame the evolution of social groups -- even civilizations -- over time, based on stressors imposed by environmental and cultural systems. Cultures themselves are examples of GST’s elements of selectivity and discrimination, creating powerful mappings shared by the people and groups that comprise the society -- which might be very different from other cultures, creating an opportunity for conflict.
• Freakonomics -- The popular Freakonomics books are regarded as insightful application of economic theory, specifically focused on incentives. But incentives are just stressors, so systems indeed respond as GST would predict. Consider a core Freakonomics example, in which Chicago teachers responded to incentives when they cheated to help students score highly on standardized tests. Freakonomics and GST agree that the situation was caused by an incentive/stressor -- performance evaluation and compensation – and drove the cheating behavior. GST goes a step further to explain why, with identical incentives, only some teachers cheated; the teachers who didn’t were able to equilibrate through adaptations and accommodations that didn’t involve breaking rules. Perhaps they had different cultural mappings, or for any number of other psychic, financial, or emotional reasons the stressor simply didn’t have enough impact to cause the cheating equilibration.
• Atoms and Energy -- To be universal, a theory has to work with the basic building blocks of the universe, and GST does.  The hierarchical nature of systems is eloquently expressed in the model of particles, atoms, elements, and molecules. These building blocks form the basis of our natural world. In biology, you see similar hierarchies of cells, tissue, organs, and organisms. This speaks to the concept of layered complexity, as systems become ever-larger aggregations of subsystems, leading to more and richer relations, as well as greater interdependency expressed as system cooperation, competition, and control.
• Politics -- I'll use the US example here, but I'm certain it translates to any political system. Many hierarchical systems make up government, from local to state to national and (nominally) world organizations such as the United Nations. At each level there are candidates, committees, and political parties. Each acts in its own interest but participates in multiple supersystems, with a set of consistent stressors (money, power, service) as well as a set of ever-changing stressors (economic, cultural, political). In one cynical but obvious example, the equilibrations (behaviors) of politicians can often be seen in clear relief by observing the stressors (lobbying and money) imposed by the economic systems whose interests are affected. And we see examples of subsystem control hazards, for example when one wing of one party of one house of congress shut down the US Federal Government in October, 2013.
• Ecology -- Nature is a beautiful example of GST forces at work. In the big picture, the earth’s ecology is a massive and stunningly complex interconnected system that equilibrates pretty well to sustain an overall balance. And that’s a good thing, because without these equilibrations our world could not exist, at least in a way that we could be here to perceive it.
       However, we’re all well aware of how relatively minor stressors can create crisis across the planet’s entire ecosysytem. Humans have caused many extinctions through destruction of habitat. Air and water have become polluted with toxins and even hormones. And the release of CO₂ into the atmosphere is driving higher average temperatures that the earth is struggling to equilibrate against.
• Music -- The highly mathematical structure of music is one strong indicator of GST at work. All music has systems of notes and scales and chords – the mapping music uses to impart meaning to sounds. Genres tend to embrace certain combinations of instruments that yield the optimal sound. And arrangements tend to follow well-developed sequences.
       But what’s most notable about music is its infinite variety, and the creativity that goes into it. A relatively small set of common elements generates amazing variety. The resulting sounds create stressors against the part of the human psychic system that selectively relates to music’s beauty. And if you’ve ever gotten lost in a symphony or found yourself pogoing to a punk rock song, you know how powerful those bonds can be.

 

Conclusion

That’s the basics of GST. As an all-encompassing theory of the universe, of course there’s lots more. And I think that’s all valuable at an academic level, but I just like the way it can alleviate the messiness of the world, and reduce the noise.

I mentioned that I learned about this in a philosophy course. There’s definitely an element of that in there – certainly there are world views that would philosophically disagree with such a stripped down approach to understanding. But GST is ultimately about logic and rationality. In the end, the benefit is whatever you get out of it. To me, it’s just a few handy tools that can help you see what’s really going on, and perhaps make the world itself a little simpler to navigate.

Things I don't understand

Thanks for bookmarking my new blog.  So far it sucks but I'm gonna keep after it. :)

I'm going to change a few things right off the bat, using my reserved right there in the fine print.  I do have more long-form things to post -- the General Systems Theory piece will be at minimum fun, and a bit mind-bending, if you're into that kind of thing.

But I'm not sticking to just long-form stuff, because I realized the advantage of a blog is its clear ownership. On public media, you risk looking like a dork if you stand on a soapbox.  But a personal blog is a place you can try that stuff, especially if you invite discussion, as I always try to do.  So that's the criterion: stuff I wouldn't post on Facebook between the food porn and puppy pix.

To sum up: I'll keep bloviating here about things I believe I understand, and be open to reasons to rethink these beliefs.

But there's another category of things, ones that I admit I simply don't understand. I invest time in trying, because logic and evidence seem to dictate that they should be different from the way they actually are.   My brain glitches when I see these incomprehensible things, and it turns out there are a lot of them.  So maybe "Things I don't understand" can be a regular blog topic if people find it interesting.

First off, I accept that there are things I'm just not wired to understand.  For example, over-the-air television feels exactly like magic to me.  But there are people who do understand it, so I don't actually need to.

No, I'm talking about things I actually have tried hard to understand, but simply cannot reconcile to logic.  Some of them are kind of strange, ok.  Many are in the areas of economics or politics, which makes sense; there are also a lot of them in the areas of roads and transportation.

For example, this one has bugged me for decades.  You see a lot of dented guard rails, smudged jersey barriers, and scarred trees, but you don't see that many accidents or dented vehicles, do you?  Even considering many vehicles over time, that doesn't quite reconcile to me. I know the math must work, based on the evidence.  But it just feels that those numbers don't match, you know?

Ok, that's a silly example.  Here's another -- one that I actually know something about and have thought through, yet still can't reconcile.

In 1988, New England Telephone was running out of numbers in the 617 area code, which covered central and eastern Massachusetts.  They were faced with two choices:

1. They could institute 10-digit dialing, and the new 508 area code would overlay 617
2. They could split 617 apart, and assign everyone outside of the Boston metro area to 508

If you think about that for even a second, you realize that overlaying the numbers is better because:
 a) Everything about 617 continues to be true
 b) 508 just extends 617 capacity
 
So with overlay, anyone with an existing 617 number (which was everyone) would see no change.  New numbers would be assigned to 508, but be in the same well-defined geographic area as 617.  Essentially, 508 and 617 become the same thing as 617 always was by itself, so your explanation is very simple.

One more critical piece of data: Overlay had the disadvantage of forcing everyone from 7-digit dialing to 10-digit dialing, but it was mathematically certain that 10-digit dialing was coming anyway.  The explosion in first, fax numbers, and then mobile numbers, was well understood by everyone in the industry.  10-digit dialing was inevitable, and it was going to be soon.

You know what happened.  They chose to split 508 out from 617.  This forced millions of people -- from Provincetown at the tip of Cape Cod, to the Quabbin area where I live (one town from area code 413), to Salisbury on the New Hampshire border -- to change the way EVERYBODY contacted them.  They needed to print up new forms, cards, stationery, etc., and change the way they were listed in myriad places (remember, no internet listings in those days).

I remember the arguments at the time.  There were sweet old ladies with rotary phones, who would be horribly inconvenienced by dialing three more numbers every time they wanted to call their families.  I'm not kidding, that was the argument against all technical and logical counterarguments.

Folks, all those old ladies are DEAD now.  And we are stuck with a completely unintelligible area code system in Massachusetts.

To make it worse, they kept doing it.  In 1997 (less than a decade later), 617 was overloaded and they split out 781 (suburban Boston ring), and 508 was overloaded and they split out 978 (northern towns).  So again, they made everyone change everything.  Those in 978 got to change for a second time.

But the final irony is that, by 2001, they apparently had somebody in charge who could grok this concept.   Overlays were added to 617 (857), 781 (339), 508 (774), and 978 (351).  Presumably it will operate this way going forward.  But who knows?  Maybe phone numbers go away altogether, subsumed by the much more flexible and comprehensive identity models of today's systems.

Anyway, those are some examples of things I don't understand.  I have some more I'll add in future posts (political primaries WTF?).  I welcome your comments or corrections.

Internet Privacy: A Free Market Solution

One of the thorniest current issues is the way companies are abusing personal privacy on the internet. There’s an arms race underway to turn our personal data into profit, and much of the current internet investment and innovation seems to be in this space.

This analysis asserts that -- from a purely economic perspective -- the current approach to internet privacy is driven by asymmetric, and in many ways perverse, economic incentives. In this post I will propose one solution that attempts to properly align those incentives, and restore market sanity -- for vendors and users alike.

The Problem

We reveal our most personal information every day on the internet; sometimes it's intentional but more often it's not. With even casual web surfing, every click and post and purchase reveals a little more about who we are, who we know, and what we believe.

The current economic incentives are driving companies to aggressively transform their most unique asset -- unprecedented access to our personal information -- into revenue. The conventional wisdom in the marketing field is that more and better information will make it easier for marketers and advertisers to separate us from our money.

Thus, every day companies are inventing new ways to capture your personal data, and analyze your activity on the internet. For example, every day companies :

• log (and often share) every click and action they can associate with you
• track you almost everywhere you browse via “tracking cookies” correlate data about you from myriad online and offline sources
• capture geolocation data to record physical movement over time
• apply sophisticated proprietary software to build a psychosocial profile of you

There’s a widespread revulsion against this pattern of increasing privacy assault. Unfortunately, this moral incentive to respect privacy is overwhelmed by the economic incentives not to.

The net result has been a kind of helpless resignation, as people are continuously told that this is the price for our shiny internet toys. As far back as 1999, Scott McNealy, then-CEO of Sun Microsystems, said, “You have zero privacy anyway. Get over it.” Following the 2001 terrorist attacks, Oracle CEO Larry Ellison, whose company later acquired Sun, said: "The privacy you're concerned about is largely an illusion. All you have to give up is your illusions, not any of your privacy."

Of course, Sun and Oracle were/are enterprise-focused companies and thus have little fear of alienating individual users of their technology. Companies like Facebook and Google, regarded as the premier data-harvesting front ends for personal information, make lots of noise about respecting user privacy, all the while acting in the completely opposite way.

The Economic Root Cause

But why does this situation exist at all? Clearly the companies believe it’s the most profitable approach, and there’s some evidence to back them up. So there are clear perceived economic incentives to abuse privacy, even in the face of the widespread feeling that what they are doing is “creepy.” And that’s why the industry will fight any effort to rein them in.

But unlike moral incentives, economic incentives don’t judge. So the issue isn’t just the existence of economic incentives that favor privacy abuse, it’s the absence of incentives to respect user privacy. Simply put, companies pay no economic penalty for abusing privacy, so they act rationally in doing so.

Based on the intense interest in the topic, however, it appears quite possible that respect for user privacy could be every bit as powerful as a competitive differentiator as a larger network, or better features. In reality, every day companies compete on all sorts of characteristics -- but it seems nobody competes on privacy.

What’s missing is a mechanism for companies to easily compete on the basis of their privacy policies. But what would that mechanism look like? And why hasn’t it happened yet?

How We Got Here

It’s important to understand the three major drivers that underly the internet privacy problem. You really need to fix all three; it’s unlikely that solving one or two would do the trick.

They are:

• What is motivating companies to abuse consumer privacy?
• What mechanism enables these privacy abuses to exist?
• Why is it so hard to correct the problem?

Motivation

As we have already discussed, the easy question to answer is the first one -- if people hate it so much, why is it happening? The answer is money. (But then, it’s always money.) And the history is quite important here.

The major internet companies, and their prospective competitors, are for-profit enterprises. There’s certainly nothing wrong with that; the whole goal of capitalism is to create wealth, in turn generating jobs, taxes, and market value.

But a successful business model on the internet has proved to be elusive for all but a handful of companies. Going back to the “dot-com” boom and crash of the early 2000s, it’s been, “Get the eyeballs now and we’ll figure out how to make money on them later.”

The monetization strategy always comes down to one thing -- advertising. There’s a certain comfort with the model that brought us previous no- or low-cost content, ranging from newspapers to radio and TV networks. But the translation to the internet isn’t as natural as it seems, for a number of reasons.

Most obviously, there are infinitely more content sources, due to the low cost of entry in building a website. This is the “long tail” at work, as many websites appeal to very narrow audiences.

The other major difference is the interactive nature of the internet itself. Computers do things with unprecedented speed and flexibility, and marketers take full advantage of that.

Traditionally, advertising has been matched to content at, or before, creation time. It's based solely on context -- that is, ads are matched to the content they sponsor. In perhaps the simplest example, sports content tends to attract ads for beer and cars, because of its predominantly male audience. Much of the mainstream internet still follows the same general model, where marketers try to find content that would appeal to specific audiences, and target ads to those audiences.

But the dynamic web can offer much more granularity, matching ads to content in real time, according to a virtually limitless number of variables. And that requires the sophisticated tools, something that long tail entities are unlikely to be able to do on their own.

Most websites don’t have scale or capacity to also sell and host their own ads. This led to the creation of ad networks, which make small payments to websites in return for the right to display ads. The actual advertisement may never have even been seen by the content creator, and the choice of ad is completely up to the ad network. (This led to the “punch the monkey” variety of terrible multimedia ad, which often has no connection at all to the website itself.)

The breakthrough was Google’s development of “Adwords,” which enabled the company to match ads to any search term. The results page included a clearly marked set of text-only ads related to the search term. Unlike the traditional “spray and pray” model of advertising, the ads only showed when a user, through search, indicated an interest in a particular topic. They weren't obtrusive and often actually helpful, so they felt like a fair trade for the services received. And the advertisers who bid against each other, via automated auction for specific search terms, were only charged when a user actually clicked on the ad. This was a real and quantifiable return on advertising investment, and quickly turned Google into a billion-dollar company.

Adsense was Google’s next innovation. This allowed the text ads to be placed on any web page, matching terms to the page contents, which Google already had in its database to support its search business. This benefited millions of websites, which no longer had to find advertisers or subscribe to ad networks, and could be reasonably certain that the ads would be relevant and relatively unobtrusive to its audience. Again, it felt like a fair deal to advertisers and users alike.

But with targeted internet advertising proving it could command high prices, the race was on to innovate further and faster. The next frontier was to not just optimize for the context of the ad, but to the individual user. Ad networks turned to the common “cookie,” a small piece of code developed to maintain state during and across user sessions. By tying the cookie to the ad network, not the hosting site, it provides the ability to follow users around the internet wherever that ad network is used (and many websites use multiple ad networks). This allows marketers to track user behavior across sites, and increases exponentially the volume of information available about each user.

And then social networks happened. Rather than needing to interpret users’ tendencies from their clicks and actions, users began telling websites explicitly... and in amazing detail. This accelerated the arms race exponentially, as Facebook, Twitter, and LinkedIn collected enough data to make marketers drool. Google+ is a direct response to this gap.

So the driver for this privacy abuse is the belief that hyper-targeting is the key to money. Or more specifically, the fear of being out-hyper-targeted by the competition, and thus being less valuable in a world of personalized advertising. That in turn is driving the abuse innovations, such as correlating user information across multiple online and offline data sets, tracking movement via mobile devices, and especially the use of high-powered analytic software to squeeze every last bit of actionable information out of that user data.

Mechanism

That brings us to the second question: What mechanism enables these privacy abuses? If it’s such a hot-button issue, with even non-privacy-obsessed people finding it objectionable, how could the problem become so big, so fast?

The answer to that is the one-sided contracts we all sign each time we interact with a website. And that’s what we do every time we click on a new “Terms of Use” -- usually without reading it. If we don’t sign the contract, we don’t get to use the companies’ services. So we hold our noses and click through.

Beneath it all, we understand those contracts -- and the privacy policies they contain -- aren’t written by our lawyers to protect us. Rather, they’re written by lawyers working on behalf of the companies, and whose economic incentives (billings) dictate that they’re structured for the companies’ benefit. Absent our own lawyers working for our benefit, the outcome is fixed.

But if you remember your seventh grade health class, any relationship where the power is held by one party exclusively is at high risk of becoming abusive. Internet privacy is no different. With all the power, and no economic incentives to respect your privacy, the internet companies are acting rationally in doing everything they can to sell your privacy to the highest bidder.

Inertia

The last question is related but different: Why is it so hard to correct the internet privacy problem? The answer lies in the sheer number and variety of contracts we sign.

Here’s an exercise. Try to think of all the Terms of Use you’ve signed over the years. Could you estimate a number? Could you remember them all? Do you have any idea what you signed up for, or which companies have better or worse privacy policies? And how many have changed since you signed them? (It’s common practice for continued use to signify acceptance of any changes in terms over time.)

And even if we all made the effort to read and understand these contracts, there’s no way we could negotiate terms for all of them. Besides, there’s little incentive for the companies to negotiate with individual users; it’s economically sane to just write one contract and say, “accept our terms or don’t use our service.”

For most people. the natural human response is to just throw your hands up and say it’s hopeless. You may not like the privacy abuse, but you like rich internet applications and interacting with your friends, so you put up with it.

It’s what Scott Adams, creator of the Dilbert comic, calls a “confusopoly.” When competitors in a market don’t want to compete (and despite what many people think, companies hate to compete because it drives down profitability), they typically try to make it so hard to understand their terms that people stop trying to. Consider wireless carrier plans or insurance policies, and you’ll see what he means.

Sure, some will try to fight back through technological means such as ad/cookie/script blockers, anonymized surfing, or “do not track” settings in their browsers (which are still optional for companies to comply with). But that’s hard, and the vast majority of non-technical users couldn’t do that effectively. Others may push for legal protections from their governments, but those are subject to intense lobbying efforts by the affected companies, as well as inconsistencies across jurisdictions.

So if technical and legal approaches won’t stop the internet privacy problem, what will? Well, in a very real sense the free market created the problem. Therefore nothing except the free market can change the equation.

A free market solution

To recap: Economic incentives are very powerful, at least for for-profit companies. The free market assumes that all participants are rational actors, and will exercise their best efforts to maximize profits.

As we’ve discussed, this is exactly what internet companies are doing when they choose to abuse our privacy. The incentives are clear: companies that demonstrate the ability to deliver deeply personalized marketing command the highest prices and profits. And the growth of these companies, in both revenue but particularly in market value, is a clear indicator that this behavior is smart.

But just because a situation exists doesn’t guarantee that it will persist. In fact, the principles of free markets dictate that imbalanced situations like the internet privacy problem will be fixed, as competitors innovate ways to better satisfy market demands. Even artificial barriers like one-sided contracts and “confusopolies” are destined to fall as markets equilibrate toward a state that best satisfies the most consumers.

Thus what’s needed is an economic incentive which rewards companies that respect user privacy. Currently, companies are unable to compete on this attribute even if they wanted to. Not that they want to, since doing so would logically lower the prices and profits they can command.


Therefore, to align the economic incentives with our personal privacy preferences, a mechanism must be put in place to enable companies to compete in that area. That mechanism must counter the perverse economic incentives currently in place, and just as importantly, neutralize the legal framework which enabled the situation to happen and persist.

The Internet Privacy Registry

To this end, I posit the creation of the Internet Privacy Registry -- a simple, free website where we can all declare our own personal privacy policies. These preferences will then be made available to internet companies via web services and application programming interfaces (APIs), so that they will be able to compete on the basis of respecting your privacy.

Created for the benefit of the public, and from the perspective of the end user, this service will enable individuals to record their preferences in a variety of privacy-related areas. It will be designed for ease of use, introducing a common set of privacy variables, with clearly defined meanings and requirements. It will cover all of the areas currently subject to abuse, from tracking and clickstream harvesting, to data retention and sharing, to profiling and analytics. And it will be flexible enough to accommodate new variables as marketers continue to innovate new ways to monetize user privacy.

To be credible and comprehensive, it will be constructed with the inputs of experts in the field of privacy, and hosted by a privacy-focused organization such as the Electronic Frontier Foundation (EFF) or like-minded entity. It could even happen as a Kickstarter project. The main point is, it needs to be regarded as an independent and unencumbered project, not another “PR tactic” as we constantly see from the internet companies, whose words are pro-privacy while their actions prove the opposite.

What are some of the straightforward settings that may be offered?

• Do not track me anywhere outside your site
• Do not combine information from other sources in your profiling of me
• Do not serve me personalized ads, beyond the context of the page/site
• Do not track or use my geographic location
• Do not retain my information for more than x days
• Do not share my information with third parties

Of course, it’s easy to imagine hundreds of individual variables, which would be unmanageable for typical web users. Therefore it will also feature a handful of “privacy profiles,” ranging from complete privacy lock-down to, well, what is happening today. Additionally, third party profiles can be applied using simple text or XML files, for example custom profiles created by the EFF or American Civil Liberties Union (ACLU).

Importantly, websites can simply declare compliance with one of these policies, so that they can easily compete on the privacy component without explicitly integrating with the Privacy Registry APIs, or do so while that engineering work takes place.

In fact, none of this requires any real technical invention. Cloud services make this a fairly simple deployment. The back-end database is little more than a user directory keyed on email address, with privacy attributes associated. The APIs will be similarly simple to deploy, for those companies that choose to support full user customization rather than just declaring support for a particular general privacy policy (and full support has advantages, as we’ll cover shortly).

The real innovation needed is in the design of the privacy variables. For it to be effective, the Internet Privacy Registry must be built with the same skill and care as the privacy policies of the internet companies. Experts in the field of privacy, as well as legal review, will be needed to prevent loopholes where companies can claim compliance, while still finding ways to abuse privacy.

How it changes economic incentives

So... How does this impact the three main factors driving internet privacy abuse? Let’s examine them.

Money -- The Internet Privacy Registry gives privacy-oriented companies a clear and powerful way to compete. By introducing a mechanism to measure compliance with user preferences, it acts as a brake on the runaway privacy problem, providing an economic counterweight which simply does not exist today.

Obviously companies are under no obligation to accept users’ personal privacy preferences, and their own terms of use will still apply. But those who don’t participate -- either through explicit support for individual user preferences, or through support of one of the general profiles -- will be in competition with those that do. This is a simple, but very real, economic incentive to counter the opposing incentives existing today.

Contracts -- If the goal is to restore the balance of power between internet companies and users, the one-sided contracts must end. Having a set of terms that is dictated from the user’s perspective achieves that.

If a user declares, “Do not track me outside of your site,” or, “Do not serve me personalized ads,” those aren’t subject to fudging. Just as users are currently stuck with words prepared by company lawyers in Terms of Use, any company that claims to support the Internet Privacy Registry agrees to the terms listed there.

In practice, this will help the companies as well as the users. Even privacy-oriented companies must devise terms of use and privacy policies, and each of these basically has to be crafted from scratch. This leads to inconsistency, ambiguity, and suspicion. Since users reasonably assume that they are being abused by many, if not all of these contracts, having clear, objective terms helps both parties.

Manageability -- Having a single place to control your personal privacy policy achieves two things. First, it ends the kind of continual tweaking that is endemic to the internet privacy policies we agree to, which is the complaint many people have whenever their preferred internet sites change policies. Just as importantly, it gives users a way to easily adjust settings when they choose to. For example, if you have chosen settings that prove to be too restrictive for the types of sites you want to use, you can tune them appropriately.

And that leads us to the next part of the conversation.

Why would this work?

The first question will be, why would companies agree to honor anyone’s personal privacy policies? After all, they currently have their own carefully crafted contracts, which align with their business strategies in ways that the Internet Privacy Registry certainly would not.

The answer, again, is free markets and competition.

Take Facebook as an example. As the undisputed king of social networks, Facebook has the largest active network of people, with the self-reinforcing dynamic that people use it because their friends do too. Because of that massive population, Facebook commands similarly massive revenue from the same marketers who want every last bit of personal information in the quest for better targeting. That, in turn, drives investment in user experience and infrastructure and also, unfortunately, the kind of privacy practices people find so objectionable.

But what if a competitor emerged, one with perhaps less evolved UI and infrastructure, but a commitment to honor personal privacy as defined by the Internet Privacy Registry? Many people, this writer included, would modify their behavior by moving to the new network, and encourage their friends to do the same. Over time, this would make Facebook less attractive to marketers, and threaten its revenue and growth.

In short, by enabling companies to compete on privacy, it forces them to do so.

Imagine a scenario where millions of users invest the few minutes it would take to register and declare a personal privacy policy. That’s a strong market message that privacy matters to people, and indicates clearly that they are willing to “vote with their feet.”

But there are other, more subtle advantages to having an independent arbiter of user privacy, even for the companies that are fueling the arms race in this area.

Let’s optimistically assume that most companies are run by people who have the same privacy concerns most of us do. They may not want to participate in the privacy abuse arms race, but if the market is rewarding competitors who do, then everyone is forced into the same behavior, in the interest of investor/shareholder value.

Put another way, strong economic incentive exists to behave badly, especially in the absence of a counter-incentive to respect user privacy. The Internet Privacy Registry changes the calculus because it mitigates the bad incentives, while creating good incentives.

A related point, which we touched on earlier, involves the technical work needed to support fully personalized privacy policies.

Say a company elects to participate by declaring support for one of the mid-range general policies. Perhaps that’s in the range of, “Don’t track me outside your site, don’t combine outside data sources in your profile of me, and don’t share my information with others.” That still leaves clickstream tracking and personalization within a domain, with suggestions and ads based on previous behaviors. That’s basically how Amazon and Google built their businesses, before the creepy stuff started happening.

That simple step goes a long way toward establishing a company as someone willing to meet reasonable expectations of user privacy. But it’s a missed opportunity, economically speaking, because it treats all users the same. It’s still less restrictive than some users want, but, as importantly, it’s less restrictive than others may choose.

If a user purposely chooses a privacy policy that is highly restrictive, he is unlikely to use a site with even a moderate policy. One may argue that it’s not a profitable user, but it’s a user, and especially on social sites, losing that user is a net loss to the network.

If a user chooses a permissive privacy policy, then that’s an “opt in” user and, according to the economic incentives prevalent today, that’s a valuable user. It’s easy to see sites competing for that user in other ways, for example with extra features or benefits.

The point is that it’s unlikely that users will all choose the most restrictive policies, or even the same policies, based on normal bell curve distributions. Having the ability to truly customize to every user’s preferences isn’t just a great competitive position, it’s also an economically wise one.

Conclusion

From a practical perspective, expecting moral outrage to stop internet privacy abuse is akin to a gazelle expecting its moral outrage to stop a lion from killing and eating it. Absent sufficient counter-incentives, we can expect the privacy abuse arms race to continue, or likely accelerate.


It may well be that the Internet Privacy Registry isn't the only approach to restoring balance, or even the best one. What is clear, however, is that legislative and technical approaches will continue to fall short, because both are outweighed by the economic incentives of large, sophisticated, well-capitalized players. Only by addressing the economic vector will we have any hope of enjoying an internet that treats our privacy in the ways that we expect and deserve.