Dylan Hunter

Was an excellent post on O’Reilly Radar by Nat Torkington reminding us in the course of designing social suggestion-based sites / software to not forget one of the most critical reasons for in the first place - serendipity / new discovery. Sometimes this is best accomplished by “more like this,”‘es and/or “people who viewed this purchased..” kinds of things, (i.e. narrowing / refinement goal-oriented) and sometimes best accomplished by purposely broadening / adjusting the scope to introduce some new variance and seeing where things progress from there.

Many years ago there was a little search startup called Direct Hit here in MA, (who earned $500M on a cheaper version of what I wanted to do, but that’s another story) who really was the first entity to hang their hat on social suggestion-based search results, knowing full well that most folks only look at results 1-3, and almost no one beyond 10, so that those results that it initially presented as 1-3 would almost always tend to stay there, whether they were the best results or not - i.e. for anyone who’s ever used Excel, an infinite circular error.

As we all took this kind of click-through analysis / adjustment functions in to become one of the ranking criteria we used, (and / or at least used to internally evaluate how good a job we were doing in delivering relevant results - with all of the appropriate mechanisms for negating bounce - i.e. clicking through on a link, saying “this isn’t what I wanted” and hopping back to the search page, etc) the import of regularly introducing different results into the mix to make sure we were actually doing a much better job of delighting folks became more and more clear then, and still holds true today.

But enough of me babbling, Nat does a superb job of describing, so go read it from him!

I’m hardly an anti-javascript zealot, (in fact, there are many times where I’ve been a strong proponent, and am definitely interested in where AJAX could go) but I do have to say I definitely find it highly irritating that when I go to a new website without javascript activated, I either get absolutely nothing, some random jumble of text, and/or can’t navigate at all. Come on, folks, javascript is supposed to _enhance_ the experience, not preclude it! And yes, as a user, given the horrible extent to which companies, (even ones with great brands) have abused javascript for use with pop-ups/-unders, it is definitely a wise idea to start with it turned off, and only enable for trusted sources!

If you’re one of the folks who went over to Firefox primarily to get away from the pop-up hell that is IE, (and/or because you believe that no one company should have the level of hegemony that MS has commanded for quite some time, especially not with the business practices that are all too tempting for any monopolist, but especially MS) you will have noticed that in the last few months, as our new friend has started getting some very strong browser penetration numbers, that some folks have started breaking through Firefox’s higher resistance to pop-ups, etc.

To get back the browsing environment that we got Firefox for in the first place, I _strongly_ recommend installing the NoScript extension, which defaults javascript off, but when you go onto a new site with javascript, it will pop a small band on the bottom of the browser to let you know that the site has javascript. You can then either choose to enable for that site either permanently or temporarily, and pretty much be back off to the races. Although a bit of a pain in the beginning, you do get used to it after a bit, and once you start seeing all the entities that are trying to do things to your computer without your knowledge/consent, you’ll never even think about browsing without it, (i.e. did you know that Sourceforge, the core of Open Source, no less, drops Tacoda tracking cookies on you when you go that site - come on, of all the sites that cater to some of the most psychotic online privacy proponents, SourceForge doing this?!).

Now, I actually understand and believe in the value of behavioral analysis, (both for search, where I first tried even before working on Alltheweb,* and now in advertising, though in the latter case, I honestly do find myself somewhat conflicted on the issue of tracking cookies - btw, in case you couldn’t guess, I also have y cookie settings set to prompt me before anything happens! ) but on a personal level, I do find myself feeling quite irritated when things happen to my PC that I didn’t _specifically_ ask to have happen, and since I’ve installed NoScript, and now seeing how many companies are trying to do all kinds of questionable things, I would _always_ suggest going to Firefox, installing NoScript, setting cookies to ionly be set by the given site, and then to have the site ask you whether it’s ok to cookie you.

I definitely _do_ wish NoScript would modify from being whitelist-only to providing a blacklist option based on popularity of folks submitting sites to be blacklisted to them, but having worked on similar technologies in a failed attempt to pull off a Safe Search version of ATW for Pax I know how this can quickly blossom into a _huge_ pain in the tail for even firms getting paid good money to solve, forget about random folks contributing personal time and effort to help out, (thank you, Giorgio!!! ).

——————–

* Was the basis for the search engine I designed while in B-School @ Babson. Yes, for those who have poked around a bit, that’s the “better version of Direct Hit before Direct Hit existed.” Taught me several very important lessons:

1) If you believe strongly enough in the value of something, don’t let yourself be dissuaded even if authority figures you respect tell you what you’re proposing is irrational / impossible. Heck, if you think about it, there’s nothing rational about Search in the first place: making not just one, but multiple copies of all the knowledge and experience of the entire planet? Most people don’t even think about how utterly insane that is, so when was any web search concept ever rational?

2) Even for highly-capital intensive ventures, (of which, Search is definitely one) you can’t allow yourself to abandon a good idea because you’re not going to get funded. Now, back in ‘97-’99, when I was going to B-School, that was the proscription:

A. Business Plan
B. Management Team**
C. VC Funding
D. GO!

Now, in ‘05, Search is definitely coming back, so yes, there are at least the glimmers of hope, where 3 yrs ago, we were all trying to figure out how to “re-cast” Search on our resumes, since there were certainly no Search companies hiring, (at least here in the Greater Boston Area) and since there was a truly sick and disgusting semi-triumphant attitude to see those crushed in the post-Bubble fall, (i.e. early 30’s VP’s). And yes, there were many of us who _were_ jack-asses, (much of the Lycos staff coming immediately to mind - hated their partners, hated each other, bragged about so-and-so having been “shit-canned” after 6 months, the median lifetime of a Lycos person back then) but there were many of us who loved the Internet for the chance to do something extraordinary, to actually have an impact on the companies we were working for, rather than just being a random cog, and yes, for sure, to make enough in doing so both to make secure lives for our families, and to ensure that we could do it again a couple of years down the road when the funding sources told us we were out of our minds yet again, (i.e. do you honestly believe that Google could have become what it is today if it had tried to launch in the funding climate post-2000? As I said, though, am definitely pleased to see interest in Search coming back up - you’ll still have to bootstrap hard, and probably be able to make a go of it for a good 12-18 mths or so, which’ll be nigh unto a killer for many of us, but it’s still infinitely better than it was, but the question on Google post-2000 still stands…). Well, at least some folks are still living the Dream, and for the rest of us, Hope dawns again!! - And this time, there _will_ be a liquidity event before the financial markets choose to crush us again, and forget B2B - yes, it’s a very easy way to get good a good bunch of cash / traffic at one shot, but the very protracted last recession should have shown us all that corporations do _not_ behave rationally to their environment, refusing to invest even as the consumer economy remained very strong. B2C - takes longer, may need to be funded from some B2B endeavors while corporations remain “positive,” but so long as you’re providing a product / service that has value, individuals _will_ continue to consume, and you _will_ be able to survive through the down times. Businesses, as now well-established largely-economically-irrational actors will not.

Whew! Enough random catharsis for today, time to get back to productive uses - I have my own company to try to bootstrap for now, and prospecting’s not getting done while I sit here babbling about ancient history!

————————-

** ‘Course, “Management Team” was always probably the hardest of those steps, especially during the up times, as anyone who was willing to shelf his idea, where he’s the CEO, for yours, where you are, probably wasn’t good enough to be on your Team, if you really wanted those “A-players” that everyone’s always spouting about. Still a huge issue for anyone who looks back into their history and sees tens / hundreds of millions, if not billions, in value fail to have been created by the companies that these folks were a part of, (i.e. how much are the folks from Excite, Lycos, AV, as well as me still kicking themselves in the tail on a nigh-unto daily basis? Ah well, get up, dust oneself off and move on, but yes, when the bills come due each month, and you’re still playing with mail float while trying to scrape by, as opposed to working on something that people the world over will truly appreciate, and which is thus truly joyful to work on, it does get hard….)

Ok, so what’s actually managed to get me out of a 10-month hiatus to actually put up another blog post? Really, podcasting, (which I’ve been falling in love with over the last 1.5-2 mths) but that’s fodder for another post, (or 2 or 12). But at this precise moment, it’s an article in the Channel Register that’s talking about Yahoo re-assembling some of the old IBM Clever guys again, with speculation toward reviving at least some portion of IBM’s old Clever project.

Why’s this so inspiring? Several reasons. I first actually read about Clever in Scientific American back in ‘98 or so. At the time, even though officially I was (and am) a business guy, I was (and am) a total tech geek, and the beauty of the system that they described was so amazingly apparent in comparison to what was out there at the time, (yes, even our friend Google, who I remember thinking was a “cheap copy” of Clever at the time - ok, “more pragmatic” copy would probably be more appropriate - Clever could take 11 mins at query time, which was, of course, ludicrous, but from deep geekdom, it was like seeing the Mona Lisa and then thinking about a poster of some other girl - the latter was much more practical, but nowhere near as breathtaking - see, told you - was and will always be, first and foremost a math geek, so I very much to this day see beauty in elegance rather than bludgeon) that I literally fell as close to in love with a search technology as one could.

Actually, not too long before the time that that article came out, I had been working on a business plan for a new kind of search engine for the business plan competition as part of my evening MBA with Babson, (back then, winning Babson’s business plan contest was an excellent way of getting real funding and being able to actually make a real company, so we all worked mighty hard on pulling it off). In essence, it was a better version of Direct Hit, before Direct Hit launched, using user clicks as the prime mechanism to improve relevance sorts, but with the addition of a strong user profiling component, so that it only would use the clicks of folks “like you” to define your sort for a given query, and of those people like someone else when they issued the same query.

Knew the marketing side of the business would be much easier, (to Direct Hit’s absolute credit, to this day there is no company’s tagline that I love better than what theirs was - “One Site, Millions of Minds” - perfection - simple, totally got the concept across - wondrous! ) so spent 6 months figuring out the tech side, i.e.:

- How much could you know about someone the first time they came to you - what things would be the best to use and to what degree - IP resolution, (removing all entries from Vienna, VA, of course; all the AOL folks) associating the person with the user group for which his search term was most populous, etc., etc.

- How, and to what level, could you refine the user’s profile given his query and click patterns, frequencies, etc., other data that you might be able to infer from offline sources, based on the data you could pull, (most notably, via IP, allowing for the error that the IP is actually where your provider connects, not you, which can radically affect the usefulness of this variable).

- How would you know when the user’s behavior suggested a subclass of whatever class you had him currently a part of vs. when you should migrate him to another segment tree, etc., etc., (hey, I told you I was a geek at heart - the business side is how I try to make things actually come about! ).

Anyway, enough of the “Dylan’s Direct Hit before Direct Hit” story for now, (though I still find it interesting that I presented the concept in Wellesley, MA, and shortly thereafter a new company was founded in Wellesley, MA, but that’s something else…. I know, I know, and Kennedy was really killed by a hellish combination of Girl Scouts and escapees from Area 51… ).

The second part of my interest (very short version) is that I was PM for Internet Search for Fast Search & Transfer from nigh unto the beginning in ‘99, (left in ‘01) which was bought out by Overture in ‘02, (I think) and thence by Yahoo, (4 mths thereafter) so if Yahoo is indeed working on bringing at least portions of Clever out of mothballs, and reunifying it with my Old Girl, that makes me happy on so many levels, (we actually did do a deal with IBM, and licensing some of Clever was discussed, but IBM decided they were going to figure out how to use by themselves, or nobody was). I truly miss the old discussions we used to have as to what defines relevance, to whom, what technologies / areas of R&D might improve, etc., etc. Awesome fun stuff to work on, and am working right now on figuring out how best to apply, (with no cash, of course - I have little-to-none, nor do I see a whole slew of VC’s dying to give me some! ).

And this time, when I want to do PFI or text P4P, (as I did in ‘00) or, much more importantly, the next thought on monetization, (since back then, I was in a good place to pull off - now, I’d have to be an idiot to go head-to-head vs. Google & Yahoo, but am sure there are plenty of holes around them) I won’t have to worry about speculation from others that doing so would isolate our B2B clients, (even after Google had done the same thing to Yahoo) even given that they had already pushed me from $4.00 per thousand queries down to $1.75 16 months later, (soon fully inverted to “I don’t pay you - you pay me $5.00 per thousand”).

$100M for the technology when Overture bought my Old Girl out, $1.5Bil when Yahoo bought Overture plus my technology 4 mths later, $23Bil+ Google IPO…. oh man, I could have been…. Stop… Not productive, except in once again reminding me that speculation belongs only as a way of educating tests in the actual open market, but it is the market, and not random speculation, that should determine what a company tries to do. And yes, I had Alltheweb.com, which we’d grown to 5.5M queries per day at that point, half of a Lycos, (when Lycos was actually impressive) so rather than costing me $1M a year in infrastructure costs to support, could have been…. Stop…. Again, would’ve, could’ve, should’ve - who cares - it’s done.

Ignore the still-sore spot, (obviously! ) read the Clever article. Hope that the guys can figure out how to use some of the better concepts from Clever such that it can respond in less than 11 mins, (i.e. beauty is certainly beauty, but it’s still better to actually make something a reality along a path and get better from there - if something only exists in a lab, it doesn’t exist) unify that with my Old Girl, (now also combined with AV & Inktomi) and bring something extraordinary about.

For me, gives me great hope - not just for Yahoo, (though, as you can tell, I still would love to see my Old Girl glow again - it’s been a bit, certainly now @ ATW.com, she’s barely even a shadow) but more importantly, in adding to the assertion that there indeed is still life in doing Search well. I have no cash, but I don’t care - at this moment, I am more pleased than I’ve been in a long time!

Now to figure out the hardest part of the solution that I’ve been trying to figure out for months, if not years - how the heck to work on, after work, while still putting the right amount of time into my wife and daughters, who are much more critical - if you find yourself lucky enough to be married to someone who was with you when you were (at least theoretically) worth a small fortune, and she stays with you when you’re once again worth nothing, without a moment of hesitation or recrimination, (other than the “don’t even talk to me about equity again!” ) hold onto her for dear life - better than one in a million. And if you further are greeted every day when you come home, regardless of whatever happened, with smiles, loving hugs and excited squeals of “Daddy!!!” you’d be a flaming idiot, (and yes, this is preemptive for an elder me, just in case) to endanger that - for any reason - ever.