Back in late March you may have noticed that this blog was shut down. In fact the entire Softduit site was down for several hours.
We never entirely isolated the issue, but basically there were 2 things going on that could be identified:

1. Wordpress 2.1, the software that runs this Maven Mapper’s Information, was leaking bandwidth severely – leaving a call open when something created ‘persistent connections’. Persistent Connections are not always bad, but they are dangerous when not used correctly.
2. I had a meta tag plugin (Autometa) that was creating an error that was not visible on the blog but creating significant havoc with my host.

When this happened, I was a bit frantic. I didn’t know a persistent connectin from a hole in my head. My host shut my site down, much to my dismay. Thinking that there might be something like a denial of service attack going on, but there was not.

Right away, I got rid of the Autometa (bye bye). It may or may not have been a big portion of the problem, but for troubleshooting reasons it was in the way and had to go.

The thing is that persistent connections are not always bad, but it was a bit beyond me to identify the good from the bad in this case.

Here is some of the information they gave me:

System administration has noted that your account appears to be using persistent connections in some of the software used on your site. Persistent connections are by and large not necessary for most software to function, and they can cause issues with your account.

To explain a bit further, persistent connections are one method PHP scripts may use to open a connection to a MySql database. Using persistent connecting is only useful in an environment with a high overhead in connecting to the MySql database itself – in your case the connection (and ‘cost’ in resources) is negligible compared to using persistent connections as all of your queries are executed immediately.

The persistent connections in this case are simply sitting idle consuming memory and may bring your site close to the predefined limits set for accounts in the shared hosting environment. If you have a database intensive site, this could make the site appear sluggish or appear to be down while the initial queries time out (this can take up to 300 seconds, depending on several factors).

You will need to review your code and see where these persistent connections are coming from, as spikes in traffic could cause your site to appear to be unavailable to your visitors.

Plus

I had to suspend your site due to the very high load that it was taking on. There were multiple IP addresses connecting to it over and over again. Some of the IP’s had over 80 connections each.

Here is what I learned.

1. After you install a plugin, check your error logs. Just because it seems to be working doesn’t mean that it really is.
2. The jury is still out on some things about Wordpress 2.1 that haven’t been documented well by the community yet, so pay attention to your bandwidth and be careful.

Note for all people new to hosting their blog on their own domain. – Be Careful!

It is likely that you have a hosting plan that allows a certain amount of bandwidth per month. If you go over that bandwidth a couple things are likely to happen:

1. Your host might shut your site down.
2. You might get an overage charge (like going over your minutes on your cell phone bill – very very scary if it happens and you are not prepared)

We are all used to getting typically hundreds to thousands of hits per day, but if your site goes into the millions of hits per day or hour you better make sure you have a good solid way of monetizing that traffic or else you are going to be in trouble.

Adsense May Not solve the Problem

You might think that Adsense will earn a tremendous amount if you see a spike in traffic (ergo you write the golden article that captures the attention of the world and that CPM number makes you rich!)

Wrong!

Google will sometimesshut down your Adsense account because they think there might be fraud going on (Google is like that ~unpredictable) . They like to see nice constant (expected/predicted) growth in your traffic. If you jump from 100 hits per day to several million hits in an hour (and your server doesn’t crash), Google may nix your account and not pay you your earnings blaming it on fraud.

Now you have a bandwidth issue, commonly referred to as a Big Fat Bill and you have no Adsense revenue and no Adsense Account. Its kind of like getting your right hand chopped off, having salt rubbed in the wound while someone slaps you in the face repeatedly with your own hand.

For about $6 million Earthlink will deploy a wireless network in San Francisco after coming to terms with the City of San Francisco.  Google is also involved.  Google will provide free access to the network at low transmission speeds.  Earthlink and others will charge a monthly fee of $21.95 and $12.95 for about 3,200 low-income residents, which begs a tangential question, “Are there only 3,200 low income residents left in San Francisco?”

The network will be privately owned by Earthlink, which was a major source of contention.  Many were concerned that the city should own the network and had other fears of privacy protection from Earthlink and Google, who will now provide additional privacy protection disclosing their policies fully.

For skeptics of the free service, users will not have to look at advertising any more than is normal on the internet already.  At just $6 million for deployment this might be the cheapest project launched by the city providing the biggest return.

Google’s blog search functionality may be starting to eke past Technorati. Google now offers users the option to get to its Blog search functionality from the main page on Google. Users can search blogs, by clicking the more button above the search box and then selecting Blogs, which is the top option in the drop down box.

Google launched the blog search option in September of 2005, but apparently didn’t realize that linking to it from their own home page would be the thing necessary to make it successful. This is a rather startling mistake in that Google is lauded as being a very smart company, but took over a year to figure this out.

Now that they are past that lesson straight out of the obvious book, they are making up time and picking up searches from key age demographics among searchers in the age group of 18-24 year olds, while Technorati’s shows a strength among searchers that are over 45 years in age.

Google Search technology took the world by storm.  The tempest was originally hidden within Yahoo! Search and later broke away to become its own company. 

As it did this, Google assumed the throne as the best way to find information on the internet quickly and efficiently.  That was years ago and many people have since learned better how to manipulate this super highway of information, and more importantly internet readers and coral the to a site to earn advertising money from them, often with Google’s help.

One of Google’s primary advantages was its advanced and continually advancing algorithms that enables the company to continually and endlessly index the internet and all of the billions of web pages that get added, removed and updated through out the months and years.  This was a major advance over Yahoo!’s original preference for editorialized directories that served to categorize information and provide a contextual description.

Jimmy Wales the founder of Wikia the company that created Wikipedia is hoping to use a type of intelligence even greater than a Google created algorithm.  Wales would like to use NAI or Non-Artificial Intelligence.  If you are reading through the pun, you have deciphered it correctly, he wants to use people. 

This almost seems like a throw back to editorialized categories, but instead of providing descriptions, this Wiki version of internet search would provide user generated context.  Instead of a single editor or small group of editors that would be responsible as experts for a topic, the model would be more Wiki like, enabling people to contribute and refine the context from all around the world in many different languages.

Wikipedia provides an online user contributed and edited encyclopedia of everything that is entered, modified and updated in real time.

Creating search tools that are modified and refined by actual people as opposed to algorithms might just push ahead of Google Search.  Google does incorporate surfing metrics from people as well as it attempts to gauge how successfully a person has been guided to a destination website.  Wikipedia Search could take a different form where a user searching for information reads not a description but actual secondary or tertiary source background on the topic with hyperlinks off to the primary and secondary sources of that information, thus educating visitors and directing them to a site in context as they read.

Think of the tool like a three year old asking questions:

Why do fish swim?

They swim because they live in the water.

Why do they live in the water?

Because they cannot breathe air directly but use gills to extract air from the water.

What are gills?

At each branch the child or searcher or questioning group, can opt to pursue what might seem like a tangent, but in actuality they are refining their understanding so that they can gain better understanding and to possibly ask better questions.

A person sometimes has some information and might ask a question that takes them into the middle of a topic.  If they do not find the context to show them that they are learning about the process half way through, they could miss a crucial step.  Like a skydiver that misses the step 1. Pull rip chord and jumps directly to the instructions on navigating an inflated parachute and how to land.

A Wiki Search service might guide us searchers safely to Earth with a better comprehension of the topic that we are researching, and prevent the befuddlement that comes from a search result that tells us that the answer to everything in the Universe is 42.

Its a Saturday and low on my priorities today is serious writing.  I need a quick tech fill however and hopped over to Digg where I came across a great story from a writer that works for Google debunking the myth that Google works for or in collaboration with the CIA.

Now the article doesn’t really debunk anything with proof, the debunkation buck stops short of the mark and sidetracks on a very funny dialogue between a Google Public Relations Person and a Google Founder.

From a personal perspective, I don’t care one way or the other.  Both groups will do what they have to do in order to get ahead.

Could Google have been cooperating with the US Government when they moved their Chinese user database out of China and out of the hands of the Chinese government and Red Army?

Sure such a move was good for corporate relations, but it would have been good for the US government too.

Of Course Google had previously agreed to censor searches in China.

So on an evil scale of 1 to 10

Google might see giving search records to the Chinese government as an 8, but censoring information from Chinese citizens is only evil at about a 5 or 6 and advertising for russian dating women at a 0.5 to a -2, but the Google culture might even see importing Russian brides as a public service for the brides and the nerds that bring them to the states so maybe its a -10 on the evil scale.   Either way they see it that helping the Chinese Government directly is more evil than withholding the raw information from their Chinese customers.  Its a corporate dilemma worthy of Solomon and worked out by mere mortal Googlions.

Playing into the hands and designs of the CIA and or some other group yet to be named by conspiracy theorists is off the scales (you be the judge of which direction it is going off the scales).  Then again is Google being devilishly smart, kind of like The Firm Character, Mitch (aka Google) has secreted away its database some where in a safe harbor where no government can touch it, not the Chinese and not the US.  Its good for the CIA the Chinese can’t touch it but may not be helping the US stay the course in the War on Terror.

But then again the NSA has been working on systems and codes for a lot longer than Google has been putting together algorithms.   Google may know how to get the right search result most of the time off the internet but I’ll bet you dollars to donuts the NSA knows how to get information out of a Google DB when they need it, safe harbor or not.

[insert maniacal laugh here]

Google disclosed that it has reserved $187 million from the $1.6b purchase price of YouTube.  The funds are being set aside for potential obligations that may arise as a result of liabilities.

This War Chest will obviously attract the attention of copyright owners hoping to regain losses from the exploitation of their intellectual property.  However, just because the money has been reserved does not mean that this is going to be easy picking fruit for greedy lawyers.

The truth is that this is not only the bounty but the defense as well.  In any potential conflict Google will have to weigh the merits and cost benefits of using the fund to fight off lawsuits versus potentially paying settlements with the funds.  Any legal attack against Google is going to need either a big war chest of their own, or they will need a successful prior suit against Google to succeed.

As yet, no one is ponying up to engage in a shoot out yet.

The Wall Street Journal ran a nonchalant 6 sentence article(B5) about copyright claims filed against the Google Video Service today on 11-9-06.  The Claims were disclosed by Google in its recent quarterly report filed the day before.

Certain entities have also filed copyright claims against us, alleging that features of certain of our products, including Google Web Search, Google News, Google Video, Google Image Search, and Google Book Search, infringe their rights. In addition, our planned acquisition of YouTube may also subject us to additional copyright claims upon the closing of the transaction. Adverse results in these lawsuits may include awards of damages and may also result in, or even compel, a change in our business practices, which could result in a loss of revenue for us or otherwise harm our business.

Google not being a company about disclosing any more than necessary remained true to their tactics and did not reveal the source of the claims.

Speculation

This raises the specter that the YouTube purchase will be a large target for additional claims.  Google is no stranger to copyright claims and has fought them on almost every major service they have offered from Web Search to Google news to books and images.

Additional story were some what delayed(showed up several hours after the WSJ hit the streets) in catching on but they did catch on and speculation has questioned whether Google has established a ‘$500 million YouTube defense fund‘ which was promptly rejected by Google’s Chief Executive.

Microsoft is working hard to catch up with Google. We have to wonder if they will be able to succeed. After all, Microsoft has a terrible track record of throwing money at a problem in an attempt to grab the number 1 or number 2 position. Everyone has witnessed how poorly Microsoft’s efforts paid off when they attempted to bump IBM’s OS2 and Apple’s OS out of the market, not to mention poor showings in forays with Netscape navigator, and Real Player. Microsoft obviously tanked in all those matchups. . . .Well actually they didn’t, they took market leads in all areas. They have been much less successful with web design tools as Frontpage has faired miserably against Dreamweaver, but MS has a solution to that as well with Microsoft Expression, which is even offered Free through the end of the year!

Currently Microsoft is number 3 for ad revenues behind Google and Yahoo!. They would like to catch up in advertising and in search. They are hoping to leverage their presence in mobile phones, gaming systems, and Digital television to help take share by creating a bigger market. Google may hold a lead in web browsing alone, but expand that out to advertising in gaming consoles and Microsoft has an obvious advantage with Xbox 360 (Oh and they are trying to beat Sony and Nintendo in that market too.)

Microsoft is often lauded as being slow to develop and poor at creating great new applications, however they are great at playing catch up! They are also very good at waging a multi-front assault on many different competitors at one time.

Microsoft is rapidly turning itself into the General Electric of Software and Internet technology, building out many very broad and increasingly very deep markets.

Check out the article by ZDNet regarding Google’s move into Digital Video Recorders (DVR) and DVR communities and advertising opportunities.

Google seeks employees for TV service by ZDNet’s Garett Rogers — As the Radioactive Yak reports, Google has started accepting resume’s for an Interactive TV Product Manager in the Mountain View and software engineers experienced in “emerging TV standards” at the Mountain View and London locations.Google owns some domain names that seem to back up this speculation: googledvr.net and googledvr.org. They currently do not own the [...]

Impact Assessment:

Picture a DVR environment where you can truly find anything your are looking at any time day or night 365 days per year. If we were to speculate on the current major players in the market from set top box manufacturer’s, to the Cable Industry, to specialists like TiVo, we run into a common shortcomming.

No one has yet found a way to make it easy for consumer to get access to everything, through a remote control, easily and quickly.

Google has established itself based on the concept that you can ‘google’ the internet and find anything through a Google search enginge. They are working to expand into new areas from books, to video, music, pictures and more.

It would be a natural fit for google to build out a better infrastructure for current devices to work with. Which hardware manufacturer will line up first to have the “Powered by Google” logo branded all over their device and packaging?

Its interesting to note that Google one of the newest major corporations reshaping the world of advertising has reeled in Verizon Communications to resell Google AdWords at a local level.

Small Business used to purchasing ads in Verizon’s Superpages (yellow pages) will now be coached by Verizon agents on how to place their ads with keywords in Google’s AdWords.

AdWords is a service provided by Google where advertisers bid online to place their advertisment according to the keywords or page viewers most likely to be associated with the ad. They compete for placement according to others bidding on the same keywords or online viewers. Highest bidder gets the ad.

Its not mentioned specifically, but the article hints that these local placements will be geographically restricted somehow.

Verizon online directory in ad deal with Google – Yahoo! News:
“‘We’re pleased Verizon is an authorized AdWords reseller and helping local small businesses take advantage of the opportunities of search advertising,’ Google said in a statement.’”