Sharing's Downfall: Dogma

I came upon this article in the WSJ the other day.

The premise is that the new Einsteins will be born from sharing...  sharing, I guess, of scientific work, ideas, insights of one form or another.

Now this is a very interesting comment on science.

The premise is that science and mathematical problems are better solved in groups, e.g., the "polymath problem" (see this).

The basic idea is that someone picked an unsolved mathematical problem, created a blog, and after a period of weeks or months the problem solved by a disparate collection of people from high school students to professional mathematicians - all in a wonderfully collaborative manner.

I commented on this a while ago in a post on "Crowd Sourcing" I wrote.

Now in the case of specific unsolved mathematical problems the internet today offers access to a large audience of qualified folks to work on the problem.  I use the term qualified here because mathematical problems are sort of unique in this regard.  Interest in unsolved mathematical problems tend to cross a lot of boundaries and there are people out their toiling on there own approaches who are generally off the radar of professional mathematicians with the same interest but which may have something to contribute.

So qualified here means that someone understand the mathematical model of proofs, notation and so on and has developed some small or otherwise seemingly useless insight they have this is non-the-less real in the sense that it may be something no one else has figured out relative to the "big picture" of a solution.

The problem here is that many things are best not done by professionals.

An example I run into all the time are compilers - software programs that take a language humans can read and convert it into a form for machines.

Twenty years ago there were no "open source" or collaborative compilers.  Today virtually all compilers are collaboratively built save for those provided by Microsoft (.NET).

Sadly virtually all compilers today suck.

And I have to say that in terms of functionality, ease of use, correctness and reliability Microsoft is the winner hands down.

Why?

Because their software tools are written by a small group of dedicated professionals whose only job is to build compilers.

The article claims that somehow the new "Einstein" will be some collaborative deal.

But, by his very nature, Einstein took all the standard scientific dogma and turned it on its head to create his theories.

Groups, by nature, reject discarding dogma because to do so would leave the group member open to criticism and ridicule by its members.

I remember distinct about the time I was creating pdfExpress (see the linked blogs for backup on this).

Basically pdfExpress operated in a completely new way than other PDF tools (and still does today).  The whole point of its tremendous success was that it did things with PDF in a way which no other product did.  I embodied a single insight I made early on with PDFs: that the files where data as much as they were graphical.  pdfExpress manipulates PDF files as data, not as graphical entities, and therefore can do things that other products cannot.

At the time I created it and initially went about selling it I faced a lot of ridicule and "shunning" from the PDF elite who created products that followed standard PDF dogma.  It was so fast that people did not even believe that it was working - yet it did and it does.

To create it I literally when though an almost fanatical "madness" and depression.  It took about eight weeks or so of total focus to get the first version working.  I still remember sitting in my office with a guy that worked for me:

"It's done!" I said.

"Okay, let's try it," said my employee.

I typed in the command line to run a test file that took about five minutes to run as an Acrobat 3.0 plug-in.

I hit enter.

After a second the prompt came back.

"Did it work?" the employee ask.

We thought it must have crashed or failed.

We opened the output file and to our astonishment it had worked!

The "madness" I went though had paid off.  I knew that no one else in the world had made the insight I had made.  It was totally outside the standard dogma and standard PDF philosophy.

The real question here is could this have been done in a group setting?

After pondering this I have to say I think not.  My insight came from a flash I still remember some fifteen years later.  From that point on I could only attempt to get my idea to work - I believed it would work - I thought it would work - but the only way to try it out was to do it myself.

What if I had posted it on a blog for "crowd sourcing?"

Subsequent treatment by the same folks that never understood the product when it actually was commercially viable tells me that the blog post would have been ignored. 

Why would those steeped in the standard dogma embrace something that turns their world on their head?

This is the real issue facing science and "crowd sourcing."

Most of science is a facebook-like process of reaching consensus.  Just like a "crowd sourced" compiler its not as good as something done by a competent pro.

Advances from my experience require new paradigms because typically the old dogma can carry you only so far.

So in some sense this "crowd source" model requires that everyone share the comma dogma.

In general someone with a totally different perception of the problem will not necessarily be able to convince others to follow them.  I know that I could not (with those steeped in dogma).  On the other hand my solution worked so well that customers could not ignore it.

So the "proof" for me was the ability to help customers that no on else could.

I think that while the notion of a "shared Einstein" sounds wonderful in fact Einstein did what he did because he followed his own path.  Would Einsteins "sourcing" buddies recognize his insight?  What if it meant that their beliefs and perhaps life's work would be invalidated?

I believe this whole notion of "crowd source" success actually limits the amount of true insight and veers the project invariably toward one based on standard dogma.

Einstein worked alone because he knew what his peers would have had to say about his work before it was finished.

After it was completed it spoke for itself.

I have to say that my experience falls into the same category - dogmatists don't want a paradigm-shifting insight to upset their apple carts...

What if Motzart had to develop his sound in a "crowd sourced" environment?

What about Bach?

What if Einstein's peers had gotten hold of his work before it was completed?

No, I think "crowd sourcing" is generally a kind of intellectual socialism that brings true genius and insight down to the lowest common denominator.

Crowd sourcing does not take into account the participant's vested interests - good or bad.