I found this interview, by Steven Levy, at Wired regarding this project. Its with two engineers (Amit Singhal, Google Fellow, and Matt Cutts, Principal Engineer) that worked on the "problem" and came up with a solution.
They talk about how they updated the Google "indexing" processing in 2009 - indexing is where the Google engine goes out and searches through all the websites it can find looking for links. Google rates sites by how many links point to it. The more links that point to your site the more "relevant" your site supposedly is.
Google also matches text to the links.
So somewhere inside Google there's a mapping, for example, of the phrase "Lady Gaga Meat Dress" to a number of appropriate sites. Since Google can't know everything everyone will want to look up they instead keep separate links to phrases like "Lady Gaga", "meat", "dress" and so on and combine them (for example by seeing which sites are linked to all three of these phrases) and displaying those results.
So the problem Google faces is that the number of links pointing to sites is not necessarily a good indicator of "relevance".
So I, as an entrepreneur, might rent out a lot of cheap server space and create a bunch of "faux" content linking to my web site. This will make Google believe that my site has a lot of links pointing to it therefore in Google's eyes makig it relevant. So if I sell "fooma widgets" and I have created thousands of faux links to my site when you type "fooma widgets" into Google my site will come out on top.
If my competitors lack my IT savvy and cannot duplicate or exceed my trickery then they will lose out on ads.
So Google, in their wisdom, now attempts to weed out the "shallow content" sites. That is, when the Google search engine scours the web for links they want to eliminate links from sites that they believe don't really offer anything relevant (whatever that might mean). So if their new search engine happens upon my thousands of faux sites linking to my "fooma widget" page they don't want them to be considered or considered with the same weight as other sites linking to my page.
As Singhal says in the interview "That’s a very, very hard problem that we haven’t solved, and it’s an ongoing evolution how to solve that problem."
At issue here is the Google concept of relevance. The basic idea that Google started with - that links to a site measure its relevance is a deeply flawed one.
1. Google, since it does not attempt to "understand" the site, cannot really know what, if any, relevance links from that site might have. If I created links to my own pages why does Google get to judge that this is not relevant?
2. Without "understanding" of the pages containing links Google is working with correlation and not causation. Correlation merely means that things like "lady gaga", "meat", and "dress" happen to occur and result in links to some sites. But it cannot tell of a bunch if foolish children were linking to these sights or whether someone with knowledge and insight was.
Google doesn't and cannot care.
Now in the case of Lady Gaga's Meat Dress it really doesn't matter. But reading the interview further we see some more troubling comments.
After Google adjusted the search engine Cutts got this email from Google user according to the article: "Hey, a couple months ago, I was worried that my daughter had pediatric multiple sclerosis, and the content farms were ranking above government sites,” Now, she said, the government sites are ranking higher. So I just wanted to write and say thank you."
And here is the problem. Exactly why are the "government" sites more relevant? Relevant in what way? Why do Cutts and Singhal get to "decide" this? What if the girl dies because Google put up a bogus site? What if the government is plain wrong and someone has found a cure that is ranked much lower?
(Google won't search for porn either - even though its protected free speech - at least in the US. You might say "hey -- that's great! I don't want my kids to see that..." - but what if you're child is involved in the industry and goes missing? Now you simply can't use Google to help you search for them...)
No, sadly this is social engineering and censorship at its absolute worst. Tinkering with things so the "right" answers come out in the judgment of these two.
What if this tinkering were done in a country like, say Egypt or Yemen, to adjust the search results so they were "right"?
What if they weren't right and people died?
The basic problem is that since Google makes money selling ads on the web they face pressure from their paying ad users to ensure that no one is out "gaming' the system. Cutts and Singhal say that money is absolutely not the motivating factor.
One probably could believe that no one told them out and out what to do and why... But who is paying their salary? Why are they paying their salary?
Long ago, when Google first started, people realized that they could just sit at their desk and Google away for their competitors sites. Each time the competitor's site came up on a search the competitor was dinged for whatever the Google ad cost was for the competitor, say $0.10 USD. So if I had my staff of ten or twenty do that all day for a few days I could cause my competitor to owe Google a lot of money.
So Google, in their infinite technical wisdom, had to create a means to defeat that process. Then users figured out more things to do, and Google figured out a way to beat them, and on and on...
Leaving a very flawed system which Google continually has to rationalize as "good".
(This is kind of like what ebay has turned into - a world of its own where pricing is based on the ebay world of who will likely buy it. Can I buy it cheaper and better outside of ebay - often yes. But inside ebay there is an artificial world of prices for things which ebayer's accept... Ebay is full of the same scamers and gamers as well.)
Unfortunately most people do not and probably will never understand these issues. They will simply Google for "cheapest high healed shoes" or "most popular watch" or whatever and "trust" the results Google provides.
Sadly Google displaced a lot of good search sites which showed the good, the bad and the ugly. Like Alta Vista. This was a really good search site - it didn't sell anything and so the results were what the results were. (Eventually Google hired one of its creators and then fired him right before his stock options kicked in - allegedly because he was too old... but that's another story.)
But now its too late... Google is an icon of search wonder.
But like the old saying goes: "The bigger they are the harder they fall..."
No comments:
Post a Comment