====================================================== All the tricks of the search engine trade and more ... ====================================================== vol. 4/issue 019 _ _ ___ __ __ _ | \ | | __/ / /(TM) / _| _ _ | \ | ||_\_\ / /_____ / /_ __ _ ___ | |___ _____| \| | __/ \/\/ / ___| | _/ _` | \| / - \/ _ _ \|\ ||__\ /\ / \ ___ \ |_| \ _,_|_|\_|_\___/_| | |_|| \_ |___/\/ \/ |_____/ ====================================================== 2003-06-06 --- Sent by subscription only! Circulation: 29,645+ very special readers, and growing ====================================================== < mailto:fantomNews@fantomaster.com > ------------------------------------------------------ This newsletter may be freely redistributed by email in its unedited form. Please do share it with others! ------------------------------------------------------ SPAM FILTER ALERT: If your mail provider or your e-mail client software is protected by anti-spam software such as SpamAssassin, Spamnix, etc. make sure to whitelist messages from < fantomNews@fantomaster.com >! ------------------------------------------------------ **************************************** * Our problems are mostly behind us. * * What we have to do now is get rid of * * the solutions. * **************************************** ------------- IN THIS ISSUE ------------- <<<<<<<<<<<<<<<<<<<<<< fantomAd >>>>>>>>>>>>>>>>>>>>>> "This is a truly incredible SEO daisy cutter!" ------------------------------------------------------ fantomas shadowMaker(TM) - the most powerful tool SEO Cloaking has ever seen. <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> = News Flash: New Prices From June 15 - Buy Now To Save Money! = fantomTip: At Last: The Really Professional Ranking Manager = Independent Study: Overture Beats Google AdWords Flat = Chris Ridings: Stop Words. Huh? Why? What? = fantomTip: Dynamic Linking For Improved PageRank = Google AdWords - Inconsistency or Censorship? = Module mod_rewrite: RewriteMap - advanced directive = The Unfriendly Future: Putting Google In Perspective <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> THE ULTIMATE SEARCH ENGINE BOOK - And why you shouldn't miss out on this one <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> = fantomTip GoogleGuy He Say = fantomSpot: Featured SE Site "Feedster - Thriving On RSS Feeds" <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> FASTEST SUBMISSION - FASTEST RESULTS Don't wait for ages - get instant qualified search engine traffic NOW! Guaranteed 48 hour inclusion to Altavista, HotBot, Inktomi, Looksmart, MSN, whatUseek, WebWombat, Slider, Corporate Yahoo and other search engines. <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> = SMBmeta - The Coming Alternative To Meta Tags? = fantomSpot: Featured SE Site "Kiwi PPC Goes Live" = The Search Engine Grapevine: News 'n Stuff = Classified Ads = Guest Columnist Gord Hotchkiss: "The Search Engine End Game" = Guest Columnist Dr Elwyn Jenkins: "Trend: Invest in Future Google Keywords" = How to Be Featured as Our Guest Columnist = fantomAd "Spying On the Spiders: Our Landmark Online Service" <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> KEEP ON TRACKIN'! Real-life search terms are YOUR instant gold mine <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> = Spider IPs: Lots of New Google Spiders Detected Plus - New Wisenut Spiders Plus - New Ask Jeeves/Teoma Spiders = Subscribe/Unsubscribe Information <<<<<<<<<<<<<<<<<<<<<< fantomAd >>>>>>>>>>>>>>>>>>>>>> "This is a truly incredible SEO daisy cutter!" ------------------------------------------------------ fantomas shadowMaker(TM) - the world's first fully automatic Shadow Domain Generator, the most powerful tool SEO Cloaking has ever seen! Instant Cloaking - another webmaster dream come true. Get it here before your competitors do: < http://fantomaster.com/fashadowmaker0.html > Note: there'll be another price hike for this and other products of ours from June to compensate for the dollar-euro exchange rate, so make sure you buy now! <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++news flash+++news flash+++news flash+++news flash++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++ New Prices From June 15 - Buy Now To Save Money! ------------------------------------------------------ (rt) As many of you will be aware, the ongoing massive appreciation of the Euro against the U.S. dollar has effectively slashed profitability for Euro focused businesses by appr. 40% over the year. Being based in Belgium, this has naturally affected us as well. Moreover, we're getting an increasing number of clients who require lots of individual technical support - an increase of appr. 33% compared to a year ago. Both these factors call for a fundamental review of our marketing philosophy in general and our pricing structure in particular. So here's we're going to do. ---------------------------- Effective June 15, 2003 (hopefully in synch with the launch of our new, completely revamped site) our product prices will be adjusted to accomodate the new market situation. We will also streamline our offers by dropping retail sales of some products, integrating them into our extremely cost-effective suites. This will increase overall transparency and usability and should help in avoiding client confusion: according to the feedback we're receiving, many customers are simply struck dumb by the wide range of products and services we're offering. So, instead of 14 products we'll only be offering 7 in future. Note that we won't actually be striking any software products from our list, and unlimited technical support will be guaranteed as before - it's just that clients will have to buy our bundled offers in future to enjoy the benefits of our world leading products. Detailed Overview ================= 1. fantomas shadowMaker(TM) Current price: $1,560.00 New price: $1,680.00 - effective 15 June, 2003 2. fantomas Super Suite(TM) Current price: $890.00 New price: $980.00 - effective 15 June, 2003 3. fantomas Webmaster Suite(TM) Current price: $295.00 New price: $369.00 - effective 15 June, 2003 4. fantomas spiderSpy(TM) - SQL 12 months subscription New product - forthcoming (Expected launch: August 2003) Introductory price: $268.00 Final price: $295.00 5. fantomas spiderSpy(TM) - Standard 12 months subscription Current price: $199.00 New price: $224.00 - effective 15 June, 2003 6. fantomas multiBlocker(TM) Current price: $169.00 New price: $224.00 - effective 15 June, 2003 7. fantomNews(TM) Premium Edition Current price: $36.00 New price: $48.00 - effective 15 June, 2003 ------------------------------------------ The following products will not be offered as individual retail versions anymore: * fantomas shadowSniper(TM) Now exclusively included in: - fantomas Webmaster Suite(TM) - fantomas Super Suite(TM) * fantomas keyMixer(TM) Now exclusively included in: - fantomas Webmaster Suite(TM) - fantomas Super Suite(TM) * fantomas subFrog PRO(TM) Now exclusively included in: - fantomas Super Suite(TM) * fantomas trackerJack PRO(TM) Now exclusively included in: - fantomas Super Suite(TM) * fantomas statsReaper(TM) Now exclusively included in: - fantomas Super Suite(TM) * fantomas logFrog PRO(TM) Now exclusively included in: - fantomas Super Suite(TM) * fantomas dateDirector(TM) Now exclusively included in: - fantomas Super Suite(TM) * fantomas spiderSpy(TM) 6 months subscription - dropped from program in favor of 12 months subscription Our freebie extras fantomas shadowCoder(TM) and fantomas mailShield(TM) will still be offered exclusively with both the fantomas Webmaster Suite(TM) and the fantomas Super Suite(TM). So take up this opportunity to get all our phantomLine(TM) products and services now at the current, cheaper price! You can do it right here: < http://www.fantomaster.com/tj.cgi?ln=fao > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++news flash+++news flash+++news flash+++news flash++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== fantomTip: At Last: The Really Professional Ranking Manager ------------------------------------------------------ From the author of both the award winning TopDog(TM) and TopDog Pro comes a new generation of website ranking software. Ranking-Manager(TM) has been engineered from the ground up to provide the most open, productive and customizable environment of any ranking software on the market today. Professional search engine optimizers, website designers, owners and developers will appreciate the ease-of-use and flexibility built into this software. The most important features and benefits ---------------------------------------- Ranking-Manager offers more features and benefits than any other ranking software on the market to date. For example: - Imports data from existing ranking software - Search engine friendly ranking and analysis - Easy-to-use step based interface for ranking websites - Unlimited urls, keywords (search terms) and projects - 19 stock reports which can be filtered, customized and styled - 5 stock charts which can also be filtered and customized - Ranking data is archived for reporting, analyzing and trending - Exports data to a variety of industry-standard formats including CVS and XLS Lots more, too many to mention here ... We at fantomaster.com have tried it out and it's immediately become the ranking software of our choice - this one beats all its competitors stone cold! So check it out here: < http://www.fantomaster.com/tj.cgi?ln=rm1 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Independent Study: Overture Beats Google AdWords Flat ------------------------------------------------------ (rt) Though the industry's been hyping Google's AdWords efforts with inordinate amounts of hoopla ever since the everybody's-darling search engine launched the program, online marketers' experience with this promo outlet have been pretty mixed, to say the least. Now, you can read it all up - subtitled "Independent research firm comScore proves that Overture delivers clicks with higher conversions and more revenue", this article (ok, so it's featured at Overture News ...) reveals the cold facts, and they're not too favorable for Google. According to a September 2002 survey conducted by comScore Media Solutions, the overall conclusion is pretty obvious: "One thousand clicks from Overture generate almost double the average revenue as 1,000 clicks from Google's product." And even though Overture may say so themselves, we for our part can confirm from several years' practical experience with both venues that, compared to Google's, their customer service is indeed better by orders of magnitude, search listings can be a whole lot longer and overall control over rankings is far greater, too. Read the full article here: < http://fantomaster.com/fn.cgi?ln=ovn01 > And in case you haven't signed up with Overture yet, why not support our free newsletter by giving them a try via this affiliate link: < http://www.fantomaster.com/tj.cgi?ln=ov1 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Chris Ridings: Stop Words. Huh? Why? What? ------------------------------------------------------ Intro: Chris Ridings is the founder and owner of Searchguild < http://www.searchguild.com/ >, a free and open community for the discussion of search engines and search engine optimization ------------------------------------------------------ Stop Words are misunderstood. They're amazingly simple but rely on some more difficult underlying concepts. Most explanations skip these concepts and draw vague conclusions. This article is different, we're going to start with the "why" and end with the "what". I'm going to try and explain the more difficult concepts as simply as possible. Consider my egotistical and completely imaginary search engine "ChrisDex". It has three pages, the text content of these pages is as follows: Page 1: ChrisDex rocks the world, move over Google Page 2: The Colin Climbers rocks world Page 3: SEO in the modern world Now if someone performs a search on ChrisDex, for example for "rocks" - then I could have a program that opens each page, looks at it and sees if the word is there. If it is, it's a valid result. I'd end up with a list of valid results that I could sort. Neat huh? Except if we got to Google's 3.75 billion documents, we'd have to open each one for each query. Two hundred years later we might have a result. Okay, I'm exaggerating a little bit - but would you be prepared to wait twenty minutes for a query result? The answer to this speed problem is something called a "reverse index". Instead of doing the work when the user makes a query, we do a bit of prepartion. Imagine how quickly we could return the results for the query "rocks" if we already had a list of all pages with the word rocks in it prepared. Our list in ChrisDex might look like this: rocks: page1, page2 As soon as the query for "rocks" is entered, ChrisDex opens this file and reads the list. Sort those and it's done. But the user could search for anything. So we need to make a similar list for every word that appears in any page in our index. Hence, in ChrisDex it looks like this: chrisdex: page1 rocks: page1, page2 the: page1, page2, page3 world: page1, page2, page3 move: page1 over: page1 google: page1 colin: page2 climbers: page2 world: page1, page2, page3 seo: page3 in: page3 modern: page3 With this list (reverse index), we can cope with any query to chrisDex really, really quickly. For example, let's do a more complicated AND query for documents that contain the word "rocks" AND the word "world". From the reverse index we read the following lines: rocks: page1, page2 world: page1, page2, page3 Then pick the ones that appear in both lists: page1, page2. If you already know anything about stop words, you're probably wondering why I put words like "the" in there. If you don't know about stop words then I'll tell you more later, for now I'm going to tell you why they're included at all! The inclusion of stop words in the index is essential to handling things like the "+" operator. Go to Google and do a search for "+the" and check the number of results. If it hadn't indexed them how would it know there were that number of results? Check the time it took. If it hadn't used a reverse index it wouldn't have been that quick. So that's categorical proof that stop words do get indexed into the reverse index. The reverse index gets a bit more complicated. That's all the data required to match documents to the user's query. But if we want to sort them in to some kind of order, we need some more bits and pieces. Most modern algorithms require the position of the word (for things like calculating the prominence) and it's formatting in comparison to the rest of the page. For the sake of simplicity, ChrisDex is just going to use the position of the word in the document. To do this we number each word in the document like so: Page 1: ChrisDex(1) rocks(2) the(3) world(4), move(5) over(6) Google(7) Page 2: The(1) Colin(2) Climbers(3) rocks(4) world(5) Page 3: SEO(1) in(2) the(3) modern(4) world(5) And store those numbers along with the pages in the reverse index. Like this: chrisdex: page1(1) rocks: page1(2), page2(4) the: page1(3), page2(1), page3(3) world: page1(4), page2(5), page3(5) move: page1(5) over: page1(6) google: page1(7) colin: page2(2) climbers: page2(3) world: page1(4), page2(5), page3(5) seo: page3(1) in: page3(2) modern: page3(4) Using the position data, ChrisDex can then tell the importance of a word to a document by how high it occurs relatively in the document (prominence). If the search is multiple keywords the position helps tell us how close the words are together (proximity). Notice something about this: even with the word "the" (which is a stop word) the index has to hold the position. This is because people could do a search with the "+" operator, for example: "+the world". Try the search on Google if you want to prove the point. It's probably time for a quick recap and simplification. What I've just demonstrated to you is that ChrisDex and any engine doesn't care whether a word is a stop word or not when building the index. They're treated just like any other word is. As far as the index is concerned, there isn't such a thing as a stop word. So what is a stop word and when do they apply? They apply only when a user makes a query to the index. Modern search engines all use an underlying OR query. In some cases the results they show are AND results but this is merely a subset of the OR query and so the processing mechanisms are the same. This means a document matches and must be processed if any of the query words occur in it. Let's say my query is: "the modern" (a silly query, but good for demonstration). Looking up in the ChrisDex index we get: the: page1(3), page2(1), page3(3) modern: page3(4) As an OR query we take every document that occurs in any one of those lists. So the list of documents that the search engine ranks is: page1, page2, page3. Taking a second look at what the pages contain: Page 1: ChrisDex rocks the world, move over Google Page 2: The Colin Climbers rocks world Page 3: SEO in the modern world It's clear that page1 and page2 don't actually have any relation to anything I might be looking for. Obviously our ranking algorithm will see to it that page3 is listed first, which is great. There's a rule about ranking algorithms though, the more documents it has to rank - the longer it takes. With ChrisDex's three page index that's fine. But, again, imagine it had Google's 3.75 billion. The time spent sorting is wasted time, because the documents will go to the bottom of the list due to their irrelevancy and the user will never get to them. If we removed the word "the" from the query: "the modern" we get a return of page3. What we can deduce, therefore, is that the word "the" is causing additional processing overhead in this query without improving results. So the search engine removes it. That's what a stop word is. It's a word that is meaningless to a query but that adds extra processing overhead. A word, that by default most engines will automatically remove if you type it in a query. There are other words that are stop words. The way to tell them is simple. Stop Words have one unique characteristic - they occur in an extremely high percentage of pages. For example, "a", "and", "of", "in" are all very very common words that will be in an extremely high percentage of pages. To summarise that, stop words only apply at query time. It's a mechanism that allows search engines to ignore words that are non-essential to the results. Essentially, this is rewriting the users query to better represent what they really wanted to ask. For example, when the query: "the modern" was entered into ChrisDex, we can presume that in the majority of cases the user didn't really want to query for every document with the word "the" in it. There's a reason it's important to demonstrate that stop words are a query time function and that the index doesn't care whether a word is a stop word or not. And that's that it clarifies your understanding of other concepts. Take proximity. Proximity is how close two words are together. Consider the following phrases: 1: SearchGuild Rocks 2: SearchGuild it Rocks 3: SearchGuild really rocks Of those words, only the word "it" qualifies as occuring so much that it is a stop word. If I have these words on three seperate pages (page1, page2, page3) in my ChrisDex index and then query for: "searchguild rocks" and if we rank purely on proximity, what would the order be? Obviously 1 comes top because the words are right next to each other. But what about 2 and 3? Both have one word seperating them, but in the case of 2 the word is "it" (a stop word). Does that make 2 come last? That's certainly a common statement amongst many people. Let's do the actual processing for ChrisDex to see. The pages are: page1: SearchGuild Rocks page2: SearchGuild it Rocks page3: SearchGuild really rocks Using the same process as before, we create a reverse index including word positions: searchguild: page1(1), page2(1), page3(1) rocks: page1(2), page2(3), page3(3) it: page2(2) really: page3(2) Run the query: "searchguild rocks" against it. searchguild: page1(1), page2(1), page3(1) rocks: page1(2), page2(3), page3(3) We can then write this as the word positions and get the difference between the word positions (the proximity) - page1: searchguild in position 1, rocks in position 2. Difference = 1 page2: searchguild in position 1, rocks in position 3. Difference = 2 page3: searchguild in position 1, rocks in position 3. Difference = 2 It's pretty clear that page2 and page3 have the same rank based on proximity. So the answer is: neither comes first (of course the engine has to choose one but it could pick either randomly). So in terms of proximity, stop words count. Looking at this the other way round shows just why this is so important. Consider the query entered into ChrisDex was: "searchguild it rocks". Which of our pages now ranks first if we base it solely on proximity? Page 2 would seem the obvious choice as it actually says "SearchGuild it Rocks". But if you do what the search engine does? At query time we look at the query: "searchguild it rocks". There aren't any "+" operators to force us to include any words. "it" is a very common word, making it a stop word. So we'll take out the stop word and rewrite the query as: "searchguild rocks". Pulling the pages and positions from the same reverse index we used for the last demonstration, we get: searchguild: page1(1), page2(1), page3(1) rocks: page1(2), page2(3), page3(3) and, therefore: page1: searchguild in position 1, rocks in position 2. Difference = 1 page2: searchguild in position 1, rocks in position 3. Difference = 2 page3: searchguild in position 1, rocks in position 3. Difference = 2 which'll look familiar :) So the highest scoring page in terms of proximity is the page 1, and not the page that has the phrase: "searchguild it rocks" on it (page2)! The moral of the story is that if I wanted to rank high for the query: "searchguild it rocks", I should, instead, optimize my pages for the phrase: "searchguild rocks". Or, in other words, when picking phrases to optimize for, always try to remove stop words. If you run the query for Google it will tell you which words it has removed as stop words underneath the query box. E.g. "when" is a very common word and was not included in your search . It's Google's way of saying that "when" is a stop word. ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== fantomTip: Dynamic Linking For Improved PageRank ------------------------------------------------------ This is a hot one you should not miss: Leslie Rohde, the developer of OptiLink (for extensive info see: < http://www.fantomaster.com/tj.cgi?ln=ol1 > has written a piece on how to actually improve and commandeer your web site pages' PR by using dynamic links. This is a highly intelligent and effective strategy you will want to know all about. So read the full article in the recent issue of the excellent "Academy Of Web Specialists Newsletter" (always worth a read - good job, Robin!), namely Leslie Rohde, "Dynamic Linking - Exploiting DHTML Techniques for Higher Ranking": < http://fantomaster.com/fn.cgi?ln=awsn01 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Google AdWords - Inconsistency or Censorship? ------------------------------------------------------ (rt) As pointed our in our article "Google Censors Cloaking and Spider Database AdWords" (see: fantomNews Flash Bulletin #011), Google took pleasure in sporting a down-the-nose approach towards our AdWords targeting search term "cloaking" - after taking us on for roughly $600, they suddenly refused to extend the campaign, their contention culminating in a message stating explicitly: "We currently do not run ads for 'cloaking'." Well, it seems that things either have changed again - or that it really depends on who is actually targeting the term "cloaking" - at the time of writing, if you run a search for this keyword at Google's: < http://fantomaster.com/fn.cgi?ln=goo04 > you'll discover two AdWords blithely advocating what you're looking for. Did Google deign to inform us of a change in their policy? No. Did they inform us that they are following double standards in their choice of advertisers? Expectably, they didn't. Well, once more we're not surprised ... ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Module mod_rewrite: RewriteMap - advanced directive ------------------------------------------------------ (bro) There's a highly powerful but little used function in module mod_rewrite we will discuss here namely the RewriteMap directive. Caveat: As this function must be configured in part from the Apache web server configuration file, any syntax error in your entries may cause the web server to crash, even to the point where you cannot restart it anymore, so take great care! To help you avert this issue, we will cover the directive in all detail to enable you to actually make full best use of it. By way of an initial summary, RewriteMap involves defining a map of keys tagged to lookups which will return a value, if existent. This map can be set up in either text or database format. In the following example we will illustrate the required configuration for a cloaking setup. To this purpose we will assume that the map in question should be set up in text format. Here's what such a configuration file might look like: botbase.txt ----------- # Date 2003-05-27 # botbase.txt -- rewriting map 216.155.200.79 spider 216.155.200.95 spider 65.214.36.156 spider This file may include: - blank lines - comment lines starting with the "#" character - lines with key value pairs The map has to be defined in the Apache configuration file "httpd.conf" with the following directive: RewriteMap botBase txt:/path_to/botbase.txt IMPORTANT: This directive may not be placed within a directory context section! The best place to include it would be right at the end of the "httpd.conf" file. That's it - at least, for the time being. Next, you will have to restart the Apache server. During startup it will read the map and load it to its cache. Now, whenever the map is requested, the Apache web server will first check whether the map has been modified since loading. If not, the server will use the cached data, else the map will be reloaded. The nice thing being that from now on you can update the map anytime without have to restart Apache. Usage of the RewriteMap function -------------------------------- To put RewriteMap to use, create a .htaccess file for the domain you are configuring this for. E.g.: .htaccess --------- RewriteEngine On Options +FollowSymlinks RewriteBase / RewriteCond ${botBase:%{REMOTE_ADDR}} ^spider$ RewriteRule ^.*\.html$ spider.html [L] Some explanations ----------------- This syntax "${MapName:LookupKey}" allows for reading the map. In our example we are assuming for a LookupKey the variable "%{REMOTE_ADDR}" which will supply us with the search engine spider's IP. After successful lookup, the value "spider" is returned. In the RewriteCond section this value will be compared with the "spider" string. If the condition is met, the RewriteRule will be executed and page "spider.html" will be called up. The point of the fairly simple example above was to demonstrate in a general manner how you can make use of the RewriteMap function. Let's now take a look at a map created in the database format DBM. This time, you will have to include the following line in your Apache server's "httpd.conf" configuration file: RewriteMap botBase dbm:/path_to/botbase.map For the purpose of this example let's assume that the file "botbase.map" includes the IPs of various search engine spiders such as those from AltaVista, Google, etc. The following Perl script "botbase2map.pl" will demonstrate how you can convert a text file into a DBM file: botbase2map.pl -------------- #!/usr/bin/perl use NDBM_File; use Fcntl; $botbasetxt_filename = 'botbase.txt'; $botbasemap_filename = 'botbase.map'; open (FILE, "$botbasetxt_filename"); @list = (); close (FILE); tie (%botbasemaps, NDBM_File, $botbasemap_filename, O_RDWR|O_CREAT, 0644); foreach $line (@list){ ($ip, $se) = split (/\s/, $line); $botbasemaps{$ip} = $se; } untie %botbasemaps; exit; --- Now, let's create another .htaccess file. .htaccess --------- RewriteEngine On Options +FollowSymlinks RewriteBase / RewriteCond ${botBase:%{REMOTE_ADDR}} ^(.*)$ RewriteCond %1 ^(altavista)$ [OR] RewriteCond %1 ^(fast)$ [OR] RewriteCond %1 ^(google)$ [OR] RewriteCond %1 ^(inktomi)$ [OR] RewriteCond %1 ^(teoma)$ RewriteRule ^.*\.html$ %1.html [L] Some explanations ----------------- This time, the RewriteCond command will not execute any string comparisons. The line ends with: "^(.*)$". In this manner we can transfer the name of the search engine spider into a variable. This variable can now in turn be used from the command lines below in order to conduct some explicit comparisons, e.g.: RewriteCond %1 ^(altavista)$ [OR] The "%1" tag will catch the variable - this is termed a "backreference". If one of these explicit conditions is met, the pertinent RewriteRule will be executed. The variable "%1" has now been defined anew and will represent any one of the following values: altavista, fast, google, inktomi, teoma. So, if we should have an AltaVista spider visiting our site, the RewriteRule will direct the spider to page "altavista.html". By contrast, a Google spider would be conducted to page "google.html", etc. These pages can be optimized in advance for different search engine spiders which will then be fed only that content you have specifically dedicated to them. Using a map in DBM format, as illustrated in our last example, will enable a dramatic boost in server performance, compared to a flat text file. This is because in a text file with, say, 10,000 entries every single entry has to be parsed via regular expression, whereas the DBM format allows for a far lower number of read operations until the desired result is returned. Applying the RewriteMap function -------------------------------- The RewriteMap function offers a vast multitude of practical applications. Here are just a few examples: - block predefined IPs or hosts - redirect predefined referrers - load balancing While we cannot cover these and other possibilities at length here, we do hope this article has served to whet your appetite for digging more thoroughly into RewriteMap! ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== fantomTip GoogleGuy He Say ------------------------------------------------------ (rt) GoogleGuy is a confirmed employee at Google's and a regular contributor to Brett Tabke's Webmasterworld forum. On this weblog page, Mark Carey keeps watch of G's statements, offering a handy way to navigate them. Check it out here: < http://fantomaster.com/fn.cgi?ln=ggs00 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== The Unfriendly Future: Putting Google In Perspective ------------------------------------------------------ (rt) Mark Hand's interview with Daniel Brandt aka Everyman (of GoogleWatch fame - see "Google's PageRank - A Critical View", fantomNews vol. 3, issue 15 < http://fantomaster.com/fn.cgi?ln=fn15 >) on Counterpunch.com puts the issue of Google's search dominance (or, as many will have it: monopoly) in a global, political perspective touching as varied a selection of issues as post 911 America, the dumbing down of the masses, the deteorization of civil liberties in general and the Freedom of Information Act in particular, the sorry state of American journalism, and, as it happens, the role of search engines in a less and less democratic Internet. Even if you should happen to disagree with Brandt, who runs, among other things, a one-man intelligence agencies watchdog operation, he's undisputably a wonderfully outspoken and articulate fellow well worth a read anytime. Here's a fairly telling quote: "If you have a 75 percent monopoly, and it's growing, and perhaps there's an IPO around the corner, you keep your mouth shut and hope for the best. That's what Google is doing. The other problem is that geeks have a poor record on social ethics, and Google is very geeky. They don't know what the word 'public interest' means; it's completely outside their frame of reference. Most of those PhDs at Google wouldn't recognize a philosophical principle if they ran over one in their SUVs. It's all binary to them - either they're gaining market share or they're losing it." He also lets on about getting kicked from the Webmaster World Forum some months back for taking a somewhat too pronounced anti-Google stance. Interesting. Read the full Counterpunch interview here: < http://fantomaster.com/fn.cgi?ln=cop01 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> THE ULTIMATE SEARCH ENGINE BOOK And why you shouldn't miss out on this one - a must-read for any webmaster! ------------------------------------------------------ This killer publication is setting an industry standard for years to come. Mike Grehan's "Search Engine Marketing: The essential best practice guide" has broken all sales records, and deservedly so! This book offers a plethora of in-depth information you won't find elsewhere. It can help you avoid numerous costly traps tied to all that dated and faulty advice circulating widely around the Internet. Mike Grehan effectively debunks many of the search engine optimization industry's fondest myths, presenting realistic, viable alternatives instead. If you're serious about efficient search engine marketing, this is definitely the best book money can buy. An indisputable must-read. Check it out here: < http://fantomaster.com/tj.cgi?ln=sem1 > <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> ====================================================== fantomSpot: Featured SE Site "Feedster - Thriving On RSS Feeds" ------------------------------------------------------ With the mounting predominance of weblogs and XML news feeds leaving an ever wider footprint on the Web, search engines specializing on just this source of information are on the uprise as well. Some of these will only index weblogs dedicated to specific topics, though, which is good for research quality but very time consuming if you're after something not gracefully covered by such strictly limited frames of reference. Feedster is focused on general search (RSS based, of course) and on top of that offers a selection of sorting features arguably making it one of the most useful tools of its kind. Check it out here: < http://fantomaster.com/fn.cgi?ln=fee0 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> FASTEST SUBMISSION - FASTEST RESULTS Don't wait for ages - get instant qualified search engine traffic NOW! ------------------------------------------------------ Guaranteed 48 hour inclusion to Altavista, HotBot, Inktomi, Looksmart, MSN, whatUseek, WebWombat, Slider, Corporate Yahoo, and other search engines. Save on regular rates, too! < http://www.fantomaster.com/tj.cgi?ln=tr0 > <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> ====================================================== SMBmeta - The Coming Alternative To Meta Tags? ------------------------------------------------------ (rt) If anything, Dan Bricklin is an IT pioneer and veteran: back in late 70s he co-invented the first spreadsheet program running on the then-new PCs. Now, he's trying to set up a new standard to replace the good old meta tags no major search engine is making any fuss about anymore due to spammers' persistent abuse. His newly developed SMBmeta (short for "Small and Medium-sized Businesses meta data") standard constitutes a proposal to tie specific web site information (e.g. what a given business is actually all about) to a domain's setup via an XML file stored at the top level of a domain that contains machine readable information about the business to which the web site is connected. This would allow webmasters to better inform search engines (should the latter actually choose to adopt the proposed standard) which web sites to look for when serving their search results. Not only would this aid small businesses in boosting their overall visibility rather than being hostage to the flukes of the search engines' ever changing ranking algorithms, it could also help search engines dramatically improve their search results with very little additional technical or financial effort. But will Bricklin be able to pull it off? While many SEO experts are sceptical, it's certainly a promising venture. ZDNet's take on the matter: < http://fantomaster.com/fn.cgi?ln=zdn03 > Read up the original SMBmeta Initiative proposal here: < http://fantomaster.com/fn.cgi?ln=trt01 > Plus, some follow up stuff: < http://fantomaster.com/fn.cgi?ln=trt02 > And here's a page linking to some third party comments: < http://fantomaster.com/fn.cgi?ln=trt03 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== fantomSpot: Featured SE Site "Kiwi PPC Goes Live" ------------------------------------------------------ Nzbase.com reports exclusively that a new New Zealand focused PPC engine was released by Altatude Marketing. Goodness knows why it had to adopt the unlikely name "iSpy", but there you are. Anyway, check it out here: < http://fantomaster.com/fn.cgi?ln=isp0 > And here's the Nzbase.com article "NZ PPC Search ...": < http://fantomaster.com/fn.cgi?ln=nzb00 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== The Search Engine Grapevine: News 'n Stuff ------------------------------------------------------ NOTE Shortlinks created with fantomas trackerJack PRO(TM): < http://fantomaster.com/fatrackjack0.html > ------------------------------------------------------ To keep this issue of fantomNews from busting your mail inbox, we've decided once more to post most Grapevine articles on a special page on our web site for a change. And yes, we again regret any inconvenience this may cause. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Find the full version of this issue's Search Engine Grapevine here: < http://fantomaster.com/fn.cgi?ln=fng19 > ------------------------------------------------------- Webcrawler Going German ----------------------- Veteran former search engine Webcrawler, now part of the Infospace property and transmuted into a meta search operation, has launched its German version. Search results are aggregated from Google, FAST, Teoma, Ask Jeeves, Espotting, Overture, and Mirago. Check it out here: < http://fantomaster.com/fn.cgi?ln=wcr0 > Searchking's Google Case Dismissed ---------------------------------- CNet reports that Searchking's case against Google, calling for a preliminary injunction due to the search engine's removal of SK's links was dismissed in accord with Google's counter motion. Read full article "Judge dismisses suit against Google" here: < http://fantomaster.com/fn.cgi?ln=cnt07 > Watching Out For Yahoo! ----------------------- In "Yahoo! Act Two", Business Week Online authors Ben Elgin and Ronald Grover take a cold hard look at what's going on at Yahoo!, the company whose perceived status was reduced to a mere has-been when the dotcom bubble burst. Since then, things have changed, and dramatically so: after signing up retired Hollywood exec Terry Semel, the Internet search icon has achieved profitability, stock quotes have soared and it seems set to mutate into yet another IT success story to the extent of copying its own legend - albeit this time based on a very pronounced and, apparently, very healthy, Old Economy approach. Every SEO/SEM should be aware of what's in the offing here: chances are, Yahoo! may impose its stamp on the search engine marketing world once again, and with a vengeance, too. So read the full article article here: < http://fantomaster.com/fn.cgi?ln=bwe02 > Northern Light Is Back In Town ------------------------------ In a surprising move, David Seuss, former CEO of Northern Light, has acquired the defunct search technology outfit from the debris of the Divine bankruptcy. While Seuss has not voiced any commitment to reinstating NL's web search functionality yet, he does want to focus on marketing the hitherto still unreleased Northern Light Enterprise Search Engine, a 64-bit enterprise search solution with high scalability based on Northern Light's taxonomy and classification capability that uses their well known clustering technology. So will they return to the web search world or not? Here's what their press release says on the matter: "Regarding the Northern Light Web search engine, which won three PC Magazine awards in a row for being best in class, Seuss declines to make firm predictions for now, but does say 'We have started discussions with interested parties about ideas for new approaches to the Web search problem, ideas that represent a collaboration between others who have done innovative work and Northern Light. These ideas may result in uniquely useful Web search services.'" Not too straightforward! However, here's what it says on their updated index page: "We are working on bringing back the Northern Light Web search engine." Read the full press release here: < http://fantomaster.com/fn.cgi?ln=nol01 > Jim Wilson - Grand Old Man of SEO - Dies ---------------------------------------- Jim Wilson, one of the fathers of search engine optimization, inventor of SearchEngineForums.com, Jimworld.com and lots of others, died of heart failure after prolonged suffering on 2003-05-06. He will be sorely missed by many who took their first steps in the ever-in-flux SEO world. Our condolences go out to his family, and our cordial "Thumbs Up!" to those who have committed themselves to take up and continue his bountiful legacy. You can find a memorial site here: < http://fantomaster.com/fn.cgi?ln=jwd01 > Standard Web Design Boosts SE Traffic By 300% --------------------------------------------- Yep - you may have had a hunch "that it were so" - now the cat's out of the bag and you can actually prove it to yourself and others: the less fancy and frilled your web site design, the better for your traffic. More info: < http://fantomaster.com/fn.cgi?ln=irt01 > Microsoft Finally Going PPC? ---------------------------- Reports indicate that MSN seems to be getting serious about rigging up its own pay-per-click program now, something the industry's expected for months now. This would probably oust Overture as their prime PPC partner, though details are still vague. More info: < http://fantomaster.com/fn.cgi?ln=cnt06 > fantomas spiderSpy(TM) Featured On Microdoc News ------------------------------------------------ For all those doubting minds out there - here's an unsolicited rave review of our fantomas spiderSpy(TM) botBase. Read full review here: < http://fantomaster.com/fn.cgi?ln=mdn01 > Don't Get Caught In Google's Cache! ----------------------------------- This Google Village piece analyzes what can happen if Google caches even those pages you uploaded only accidentally or pulled later, wishing you'd never written them. As happened recently with ComputerWorld. Read the full article here: < http://fantomaster.com/fn.cgi?ln=ggv01 > The Final Say? - Danny Sullivan On Cloaking ------------------------------------------- It remains to be seen whether Danny Sullivan's recent article will really be "Ending The Debate Over Cloaking" as its title promises. However, it seems fair to say that anyone taking up the subject in future will be well advised to read it thoroughly first. After analyzing various SEO techniques (e.g. spreading content across several focused pages, doorway pages, etc.), Danny tackles the issues surrounding the definition of cloaking proper. It is certainly enheartening to see an SEO expert of Danny Sullivan's caliber and influence take such a sober, well informed and differentiating view of cloaking technology - this is a far cry from the usual moralistic gut level "cloaking is eeeeeevil!" drivel dominating so many of today's zero sum "debates". Not that Danny advocates it in any way, he's merely being level headed about it, explaining the pros and cons in a down to earth manner, outlining the risks and pointing out the weaknesses involved. While he doesn't subscribe to the much flaunted "everyone's cloaking anyway" argument of the cloaking proponents, for him there's "approved" and "unapproved" cloaking, a typical example of the former being XML streams in paid inclusion programs. And while he nurtures no illusions about the abuse potential of cloaking, he states quite clearly, too, that "cloaking is not spam" per se. Would that more SEO proponents would decide to resign from their fond prejudices in favor of seeing matters free from hysteria! After all: "Cloaking doesn't kill search engines; spam kills search engines". If in doubt, he advises, ask the search engines whether they'll accept whatever type of cloaking you may have in mind. However, he knows that this ideal world scenario may not be really viable in actual practice: "I'd like to say all the search engines will promptly respond if asked, but they probably won't, except to those in paid inclusion programs. Still, if you've asked and ended up in trouble, then you can at least show you tried to get clarification. If you aren't an 'industrial strength' cloaker, that may help. As for working with third-party firms, understand what they are doing for you. Ask to know if there are any potential risks and get this spelled out in advance. If you aren't comfortable, walk away. Someone who's going to engage in unapproved cloaking and who is professional will tell you the risks and not try to make you think that cloaking content is something 'everyone does'. Instead, they'll explain why they do it, why they think it works and what the possible downsides will be. They'll do this because they often work with clients prepared to take those risks, so they aren't interested in trying to disguise what they are doing." All of this makes immense sense - which is why we strongly recommend you read the full article here: < http://fantomaster.com/fn.cgi?ln=sew10 > Cloaking, Deception, Search Engines, & SEO ------------------------------------------- In an e-mail exchange with Christina Buu-Hoan who published the piece "SEO Ethics: Hype and Hypocrisy?" in her weblog (see "Debunking the SEO Ethics Shmoo" item below) with yours truly, we did indulge in waxing a bit philosophical regarding cloaking and the search engines' stance on it. In the end, Christina decided to dedicate a page to yours truly's rant on the matter, so now you can take a gander at it here: < http://fantomaster.com/fn.cgi?ln=eth02 > Cloaking the Mainstream ----------------------- Brett Tabke offers a handful of amusing (and, to some extent, horrifying) examples of inadvertent cloaking on the WWW - and takes a hard, cold look at the technical imponderables involved, coming to the not too surprising conclusion: "those that complain about SE cloaking are simply overlooking how search engines work." Read "Cloaking Gone Mainstream. Languages, Agents, Doc format, - cloaking is everywhere" and follow up discussion here: < http://fantomaster.com/fn.cgi?ln=wmf08 > "It gets rough at the summit" - Google Under Fire ------------------------------------------------- In "Google's Gaggle of Problems", a Business Week article by BW Online's Technology editor Alex Salkever, you'll find a shrewd and comprehensive rundown of the current state of affairs regarding Google's standing in the search technology world. Salkever points out what most people in the SEO industry seem to have ignored to date, namely that one of Google's biggest contenders, apart from the usual suspects, is Verity who only recently bought Inktomi's corporate search technology (and, it should be noted, their customer base of no less than 2,200 clients, generating a stately $22 million in annual revenues). It's a sharks basin Google is moving in and mutiny amongst partners and clients seems to be rife. Read the full text here: < http://fantomaster.com/fn.cgi?ln=bwe01 > Brutal Search - Search Engines Positioned ----------------------------------------- Andrew Goodman's overview of the current state of the search nation is a shrewd, outspoken and well worded analysis with no holds barred. Most search engine industry professionals will probably be inclined to agree in full with his assessment: it is indeed a brutal environment out there and it's not at all certain yet who will really survive it - and to what avail. Read the full article "Differentiation Can Be Brutal in the Web Search Business" here: < http://fantomaster.com/fn.cgi?ln=tfk02 > Google The Bandwidth Monster ---------------------------- Here's a case study of a site paying around $600 on average in bandwidth charges for Goolge's "free" spidering - and a suggestion what to do about it. Our favorite quote: "Instead of just terrorizing SEO's with their webmasters pages Google should add something like: Webmasters if you configure your server make sure to enable gzip compression on it - you'll get crawled faster and save tons of dollars and make the world a better place." More info: < http://fantomaster.com/fn.cgi?ln=wmf06 > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Find the full version of this issue's Search Engine Grapevine here: < http://fantomaster.com/fn.cgi?ln=fng19 > ------------------------------------------------------ NOTE Shortlinks created with fantomas trackerJack PRO(TM): < http://fantomaster.com/fatrackjack0.html > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Classified Ads ------------------------------------------------------ Inquire here for ad rates: fnads@fantomaster.com ------------------------------------------------------ WHY META TAGS WON'T GET YOU IN THE TOP 10 - HOW TO WIN THE SEARCH ENGINE WARS Learn why meta tags alone won't get you in the top 10 - and what you can do about it. Read the industry's most subscribed newsletter and book on how to win the search engine wars: < http://fantomaster.com/tj.cgi?ln=sh > ~~~~~~~~ THE USED BOOKS PARADISE Looking for used books, out of print books, rare books, discount books, antique books? Check out the world's largest repository: < http://verygoodbook.com/vgused_t1.html > Win $100 in books by signing up for the used books newsletter, too! ~~~~~~~~ The Globalnomads Campfire Site Unlocking Freedom - the smarter way! All things offshore: anonymous banking, offshore incorporation, tips & trick's for the PT's life, and more. < http://globalnomads.com > ~~~~~~~~ COLD CASH FROM THE PRINCIPALITY! The tax haven Offshore Lottery straight from the Principality of Liechtenstein * sponsor the Red Cross + Red Crescent * 100% tax free * ultra-confidential handling * everything bona fide + state supervised • $1 Million Jackpot lump payout guaranteed! • fast online access (multilingual) * simply grab your credit card and click along – and good luck to you! < http://fantomaster.com/tj.cgi?ln=lotto > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Guest Columnist Gord Hotchkiss: "The Search Engine End Game" ------------------------------------------------------ Intro: A former award winning advertising copywriter in the radio industry, Gord Hotchkiss started his own small advertising agency in Kelowna, BC Canada in 1988. In 1996, Hotchkiss Advertising discovered the concept of search engine positioning and started offering SEO services in the same year. In 2000 Hotchkiss Advertising Inc. legally changed its name to Searchengineposition (Canada) Inc. Visit their web site here: < http://fantomaster.com/fn.cgi?ln=sep00 > ------------------------------------------------------- Enough already! I’ve been trying to write this column for the past month. Every time I got close to being finished, a major search engine would make an announcement and I’d have to go back and rewrite half the column. Realizing that this could become like painting the Brooklyn Bridge, a job that never ends, I’m cutting it off today. By the time you read this, I guarantee it will be out of date. Needless to say, there have been a lot of moves in the search engine industry over the past few months. We’ve already reported on Yahoo’s plans to buy Inktomi and Overture’s recent purchases of both AltaVista and Fast. Now it looks like Yahoo’s poised to go on another shopping trip, and speculation is that Overture could be at the top of their list! The flurry of recent activity is a little confusing and unexpected even to industry insiders. To those outside the industry, well, let’s face it, they probably haven’t given it a lot of thought. Here’s why the recent moves are important. Web search is here to stay. The industry is maturing and the major players marking their territory in anticipation of an all out turf war. Revenue models are starting to work and search engines are beginning to make money. Pay per click and paid inclusion have turned around bottom lines for the biggest players, and now that positive revenue appears possible, the competitive market is starting to consolidate. Make no mistake about it. Yahoo, Google, Overture and MSN are betting that search will turn into a multi billion dollar industry (currently, total search engine marketing revenues are about 1.5 billion). What’s happening now is analogous to a chess match entering the end game. The pawns have been sacrificed and the power pieces are now engaged in a complicated game of attack and counter attack. When you look at it in that light, the recent moves by Overture and Yahoo make a lot more sense. The Recap --------- Here’s the search engine consolidation game recap up to the present. 1998 – Lycos buys Hotbot 2000 – Ask Jeeves buys Direct Hit 2001 – Ask Jeeves buys Teoma 2002 – Looksmart buys Wisenut 2002 – Yahoo buys Inktomi 2003 – Google buys Blogging Company - 2003 – Overture buys AltaVista 2003 – Overture buys Fast 2003 – Yahoo makes $750 M bond offer The Players ----------- The recent search engine buying spree has whittled the industry down to three or four real players (the number seems to be in constant flux), with two more dark horses that have an outside chance to stake a claim to enough search territory to survive the shake out. Google ------ Google taught the search engine business a brutal lesson when it emerged out of nowhere and quickly gobbled up a major percentage of search traffic. Largely ignored by major players including AltaVista, Yahoo and Inktomi, Google quickly and quietly built a fervently loyal user base. By the time the major search engines took notice, it was already too late. Google had established its foothold and was now direct competition. Ask Jeeves purchase of Teoma and Looksmart’s acquisition of Wisenut, two promising search technologies, are proof that the industry won’t make the same mistake again. Google is in the enviable position of claiming 30% of search engine traffic through their own portal. Our own research indicates that Google usage tops 60% with more experienced Internet users. No single search engine has owned this big a chunk of the market since the beginning of the Internet. Unlike Overture’s current position, where traffic depends largely on partnerships, Google owns a sizable piece of the search market outright. In the category of short term assets, Google can also boast current partnerships with Yahoo and AOL where they provide search results. This means that Google search results are now served up to over 85% of all search engine queries. Google is bankrolling their future through the success of Adwords. These query specific text based ads appear to the right of search results on Google and advertisers pay on a pay per click basis. The recent addition of contextual Adwords gives Google another revenue stream at minimal cost for implementation. Google’s contextual ads are Adwords served on a partner’s site when the content of a specific page matches a keyphrase that has been purchased by advertisers. For more about Google’s contextual ads, read our article. Another recent move by Google throws up some puzzling scenarios for the future. I’m speaking of Google’s recent acquisition of Blogging company, Pyra Labs. The move made little sense for many search engine industry watchers. Danny Sullivan believes it’s to give Google another market for context sensitive ad distribution. He also feels Google might be succumbing to portalitis. Our own Rob Sullivan believes that it will give Google a bit more control over run away link building on blog sites, a practice known as “Google Bombing”. Google’s strategy for the end game? I’m guessing it’s a continuation of the Google strategy since day one: Enhancing the search experience for users. Google knows its core business depends solely on the quality of its search results. It’s the one thing they have that sets them apart. When the entire future of your business depends on one aspect, you’d better pay a lot of attention to it. Google always has, and I don’t see that changing. Overture/Fast/AltaVista ----------------------- While Google’s strategy has the odd black hole, its crystal clear compared to what lies ahead for the new Overture/Fast/AltaVista alliance. While we wait for Overture’s shareholders to approve the proposed purchased, speculation runs rampant about what this means for the future. Overture’s biggest problem has always been its reliance on its partner network. Currently, the vast majority of Overture’s revenues come from two of its partners, Yahoo and MSN. This gives those two an exceptionally big stick to carry into any negotiations about revenue sharing with Overture. Should one of them decide to dump Overture, the blow could cripple the PPC pioneer’s bottom line. That fact has seemed to elude share holders, who have continued to invest over the past few years, keeping Overture’s share value hovering in the respectable $25 to $35 range. Overture has always been very aware of their Achilles heel and the recent purchases of AltaVista and Fast are an attempt to put together a defensive strategy. But if you’re buying allies, you’d probably want a little stronger back up than the new Overture team. AltaVista is literally a mere shadow of the once dominant search portal. Its market share has slipped to a negligible 4.5%. But that market share is huge compared to Alltheweb.com, which features results from Fast. Here, the audience is so small it doesn’t even register on the tracking reports. So the question is: why, Overture, why? I think it’s a combination of what Overture needed and what was available to buy. Next to Google, Fast probably offered the biggest index and most relevant search results. It comes out of the box as a pretty good search solution. And Altavista does offer a recognizable, although battered, brand for internet users. Perhaps by combining the two, Overture can come out of it with one viable search portal. And that’s something that Overture desperately needs, so it can supplement its CPC search offerings and provide an all in one search solution. Also, by announcing their intentions, Overture effectively blocked a couple of potential partnership deals, including one by espotting, Overture’s major European CPC competitor. Another benefit is that the AltaVista purchase gives Overture a patent portfolio that will probably be helpful in an IP war that’s currently brewing with Google. The wild card in this is how Overture’s major search partners will react. MSN is holding their cards close to their chest and have made no announcements about how Overture’s buying spree will affect their relationship. They have announced recently that they are paying close attention to search and are throwing significant resources at building their own solution. Yahoo’s deal with Overture still has 4 years left in it, but indications are that Yahoo is also laying the groundwork for their own CPC offering. Yahoo/Inktomi ------------- Yahoo’s purchase of Inktomi this January was the event that seemed to start this recent chain reaction. Suddenly, many existing partnerships seemed to be in jeopardy and there were no guarantees for the future. The biggest question mark is what will happen with Yahoo and Google. Last year, Google’s agreement to provide search results to supplement Yahoo’s directory results was extended for an indefinite period. In January, Yahoo went to the extent of featuring the Google Results prominently and moving their own directory results out of the spotlight. With Yahoo now having access to Inktomi’s directory results, it doesn’t seem to make sense for the original search giant to continue to partner with Google. The other partnership that is in question, as I mentioned, is Overture’s deal to provide CPC listings. The deal was renewed for an unheard of 5 years in late 2002. It’s no secret that the Overture deal brought desperately needed revenue to Yahoo’s bottom line and provided huge traffic numbers to Overture. It was a true win-win situation, which probably accounted for the generous time frame of the deal. But with Yahoo’s recent hiring of a top Overture executive, indications are that Yahoo is exploring their options for their own CPC program. ------------- Breaking News ------------- A recent bond offering to finance "further acquisitions" has fueled speculation that Yahoo might be preparing to purchase Overture. This twist literally redraws the entire search landscape. If there’s truth to the rumor, then the turf war just became much more interesting. Ask Jeeves/Teoma ---------------- Ask Jeeves purchase of Teoma in 2001 was the first indication that the industry had wised up and wasn’t going to be blindsided be a new Google. Teoma was coming on strong as a new search technology that took Google’s concept of relevancy based on link structures to a new level. Teoma looked at linking between online communities of interest, using a sophisticated analysis algorithm to determine relevance based on the linking structure, popularity and content of these communities. Initial reviews were glowing and the industry buzz was that this would be the new Google. Ask Jeeves purchased the new search star and began implementing the technology into their venerable Ask Jeeves portal. Direct Hit, another Ask Jeeves purchase that for a time shared search duty on the portal, was soon dropped and Teoma took over. While Teoma’s technology is impressive, the integration into Ask Jeeves effectively nipped its growth in the bud. Ask Jeeves is still plugging along with less than 10% of all search traffic. While Teoma’s technology is impressive, it wasn’t given the chance to develop Google’s cult like following (largely built through the academic community) because of their incorporation into the Ask Jeeves brand. Let’s face it, a know it all butler isn’t exactly the sexy image needed to inspire usage with the college crowd. Looksmart/Wisenut ----------------- Like Ask Jeeves, Looksmart saw Google-like promise in the new Wisenut search technology. Betting that Wisenut would give them the crawler based index and relevancy required to compete in the evolving search marketplace, Looksmart purchased the search upstart in 2001 for an undisclosed amount. Since then, Wisenut has been moved into Looksmart’s back shop for a technical tune up and additional “work”. Apparently, the job was more involved than Looksmart first anticipated, because they have yet to unveil Wisenut on their portal. Again, like Ask Jeeves and Teoma, Looksmart’s biggest obstacle is its current lack of market share. The latest Net Ratings give Looksmart only 1.5% of all search traffic. Up in the Air ------------- In addition to the previous partnerships, there are search portals that rely on third partner technology for their search results, often from one of the main players involved in the recent buying sprees. As search companies buy one time rivals, the state of these previous partnerships are thrown into question. Lycos/Hotbot ------------ Lycos currently features paid listings from Overture and traditional search listings from Fast. Lycos partner Hotbot also features search results from Fast. To date, this partnership doesn’t seem to be affected by the current acquisitions. Lycos, with its 7% share of search traffic, appears to be a good bet to continue a consolidated partnership with the new Overture/Fast alliance. Excite ------ Excite is just featuring Overture listings now, since they declared bankruptcy in late 2001. With Overture’s purchase of Fast and AltaVista, Excite may choice to supplement paid search listings with traditional search listings from the probable Fast-AltaVista combination. MSN --- MSN is the biggest question mark for the future. Currently, MSN relies on Overture for paid search listings (supplying Overture, in turn, with a big percentage of their total traffic and revenues) and on Inktomi for traditional search. MSN, who is a major search player with 23% of all search traffic, has recently indicated that they intend to pursue their own search solutions. At this point, they haven’t indicated the exact nature of this solution, but it’s a pretty safe bet that CPC search will be part of it. If they follow Overture and Google’s lead of providing an all in one search solution, they will emerge as a major search powerhouse. As Netscape learned when Microsoft finally turned its attention to the Internet, when you begin to compete against Microsoft, you’ve already lost. AOL --- Like MSN, AOL currently has no proprietary search technology. Currently, they default to Google search results, supplemented with Google AdWords for their CPC section. It seems likely that AOL, with 14% of search traffic, will continue to rely on Google as their search provided. The Future ---------- So, what will happen over the next few months? The recent developments introduce a slate of thought provoking questions: * What are MSN’s plans for the future? * Is Overture going to retire the Fast or AltaVista brand (or both?) * Will Overture be Yahoo’s next acquisition? * How long will Google search results show on Yahoo? * What’s in the future for Overture and Yahoo? * Will Yahoo introduce its own CPC program? * Can Ask Jeeves and Looksmart survive with their anemic market shares? If you have even a passing interest in the search engine biz, these questions offer a veritable buffet of food for thought. At this point, only one thing seems for certain. The battle over the search marketplace has begun. Expect Yahoo, Overture and Google to come out swinging. --- © Copyright 2003 by Searchengineposition Inc. All Rights Reserved +++++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================================================== Guest Columnist Dr Elwyn Jenkins: "Trend: Invest in Future Google Keywords" ------------------------------------------------------ Intro: Dr Elwyn Jenkins is an independent researcher. His core topic centers on technacy, the key ability everyone must have in order to communicate using the dominant form of communication in this electronic age, The Internet. Dr Jenkins resides in Sydney, Australia and is employed by Verity Intellectual Properties Pty Ltd to disseminate information created because of his work. Through Verity Intellectual Properties Pty Ltd, Dr Jenkins also works with selected companies and organizations to increase the knowledge of technacy and its application to everyday life. Dr Jenkins' focus is on builidng information for the web and to test that information by offering it to the market for purchase. His original article can be found at: < http://fantomaster.com/fn.cgi?ln=mdn02 > ------------------------------------------------------- Keyword marketing is a favorite for search engine marketers. However, many of the keywords are highly competitive, making it nearly impossible for those with smaller web sites, and lower ranked pages, to get near the top ten or fifteen results on a search result page. The smarter keyword marketers, however, are not trying to compete on current keywords, but are seeking out new keywords as they are created online. By using the little known about date range function on Google™ search it is possible to identify new keywords and begin optimizing a web page to those keywords gaining traffic from people who are in the know about these new words. Take for example, the little known word "captology". By optimizing to this word, the few web sites who traced its introduction to the web are now gaining substantial traffic. Google adds news pages each day, particularly from weblogs where fresh information is being created each day. Using Google™ search date range function it is possible to keep track of new keywords as they are being created. The word "captology" became very popular about a week ago when a new book Persuasive Technology: Using Computers to Change What We Think and Do was reviewed and promoted online. Jakob Nielsen's web site on March 3, 2003 displayed a review of the book on "captology", and from there an additional 63 web pages were added over a period of just 5 days. Online marketers are seeking new words like this to then quickly occupy a space in Google™ so that when additional traffic is brought through advertising and marketing of such a book, traffic also is brought to their web site because they are in the right place at the right time. Over the past week, at least a dozen new keywords have been created worldwide, with the majority of these in English. The ideal new keywords to locate are those that arrive because of a new book, like the word "captology" which is a newly coined word and which is used many times over in the promotion of this book. "'Captology' is a newly coined word that describes the study of Computers As Persuasive Technologies." -- [Electronic Publications: Special Interest Groups]. Marketers selling books about "captology" are those who are first into the area with pages that are highly optimized for Google to index their page selling book(s) about the topic. Then, there are software developers who are already building pages and writing text to gain a listing that performs well in a search on the term. According to marketers engaged in this activity, who also seek to remain anonymous because they do not want others to know what they are doing, or indeed create competition for the words they are chasing, capturing keywords as they are created is an investment in the future. In the early days of a word, marketers say they get between 10 and fifteen referrals a day, which is insignificant in terms of the thousands required to sell products. However, should a word become really popular, those who have been with the term have well established pages, with hundreds of links having been created to those pages as they were amongst the earliest users of those words. These pages therefore do far better with these new terms than anyone creating a page fresh. Our sources suggest that an investment in keyword marketing of this type can take between six and ninemonths to mature, and then according to one marketer, "... some words just never get popular and fade away. That is the risk we take." Some of the new words to have been created over the last short while include, "weblog" credited to Jorn Barger, "blogstreaming" credited to Lloyd Wood, and "warchalking" credited to Matt Jones. [See: New Word of the Day]. There are many others that have become popular such as "googlewhacking", googlism", and many other "google words". Says one marketer, " ... a key set of words surrounds Google and we are fortunate to be on one of these that took off mid last year and has given us more than 150,000 referrals a month since we optimized to that word. When we first started using the term, we got no more than 5 referrals a week." How are these new words found? Simply, it seems, read weblogs avidly, keep an ear to the online news, the people who invent stuff and people who are being inventive with words and search fresh listings in Google using words like "coined", "new word", "change thinking" and other like that to see what comes up. Serious marketers are investing time and energy to be optimized to keywords when they become popular. --- © Copyright 2003 by Elwyn Jenkins. All Rights Reserved +++++++++++++++++++++++++++++++++++++++++++++++++++++++ ----------------------------------------- How to Be Featured as Our Guest Columnist ----------------------------------------- While we welcome all pertinent contributions, we strongly suggest you read our "Article Submission Authors Guidelines" before submitting an article. You can request them by email from here: < fnsubmissionguide@fantomaster.com > < mailto:fnsubmissionguide@fantomaster.com > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> fantomAd -------- SPYING ON THE SPIDERS: Our Landmark Online Service! Featuring the world's most comprehensive database of search engine IPs (spiders and others), the fantomas spiderSpy(TM) is an inexorable prerequisite for professional SE research, spider development, cloaking technology and more. While the database is still very much in its initial expansion phase, it features literally thousands of spiders, being updated every 6 hours. Take advantage of our current ultra cool introductory price and sign up for the service at an annual fee way below what we will very soon have to charge for a mere month's access! See which search engines are currently featured: < http://fantomaster.com/fasvsspy01.html > <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<<<<< Sponsorship Notice >>>>>>>>>>>>>>>>> KEEP ON TRACKIN'! Real-life search terms are YOUR instant gold mine. Get realistic, get visited! ------------------------------------------------------ Merely fantasizing about which search terms people may actually use to find your site is wishful thinking: it will effectively kill your marketing efforts and drain your resources without significant cash returns. So track real-life search phrases and keywords, get free reports on what people are really searching for! Adjust your marketing to realistic search terms. Presenting the Net's best search term reference service ... but read on here: http://fantomaster.com/fawordtrack.html <<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>> ====================================================== Spider IPs: Lots of New Google Spiders Detected Plus - New Wisenut Spiders Plus - New Ask Jeeves/Teoma Spiders ------------------------------------------------------ (bro) Here's a verified list of new Google spiders: ##UA Googlebot/2.1 (http://www.googlebot.com/bot.html) #crawl23.googlebot.com #64.68.80.34 ##UA Googlebot/2.1 (http://www.googlebot.com/bot.html) #crawl23.googlebot.com #64.68.80.35 ##UA Googlebot/2.1 (http://www.googlebot.com/bot.html) #crawl23.googlebot.com #64.68.80.37 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Many more newly verified Google spiders are featured in our fantomNews Premium Edition! Sign up now at: < http://fantomaster.com/fn.cgi?ln=fneo > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here's a verified list of new Wisenut spiders: #UA Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com) an-zyborg1.looksmart.com 209.249.67.101 #UA Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com) an-zyborg5.looksmart.com 209.249.67.105 #UA Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com) an-zyborg9.looksmart.com 209.249.67.109 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Many more newly verified Wisenut spiders are featured in our fantomNews Premium Edition! Sign up now at: < http://fantomaster.com/fn.cgi?ln=fneo > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ And finally, a verified list of new Ask Jeeves/Teoma spiders: #UA Mozilla/2.0 (compatible; Ask Jeeves/Teoma) egspd420.teoma.com 65.214.36.110 #UA Mozilla/2.0 (compatible; Ask Jeeves/Teoma) egspd421.teoma.com 65.214.36.111 #UA Mozilla/2.0 (compatible; Ask Jeeves/Teoma) egspd422.teoma.com 65.214.36.112 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Many more newly verified Ask Jeeves/Teoma spiders are featured in our fantomNews Premium Edition! Sign up now at: < http://fantomaster.com/fn.cgi?ln=fneo > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Source: fantomas spiderSpy(TM) < http://fantomaster.com/faserv0.html > ++++++++++++++++++++++++++++++++++++++++++++++++++++++ ----------- fN Archives ----------- Review or download previous issues of fN here: < http://fantomaster.com/fantomnews.html#previous > For a contents list of all fN issues check: < http://fantomaster.com/fantomnews.html#content > Archived issues of the fantomFlash Bulletin: < http://fantomaster.com/fantomnews.html#fflash > -------------------------- Articles Reprinting Rights -------------------------- Free fN articles for reprinting and/or presentation on your web site are available here (please check the republication terms!): < http://fantomaster.com/faarticles0.html > -------- Ad Rates -------- Inquire here for Media Data and Rates: < mailto:fnads@fantomaster.com > ---------------- How to Subscribe ---------------- Simply go to: < http://www.fantomaster.com/fasmbase0.html > and click the submit button, giving your email address - that's all! ------------------ How to Unsubscribe ------------------ You must be kidding! :-) But seriously: sorry to miss you ... Simply go to: < http://www.fantomaster.com/fasmbase0.html > and click the "unsubscribe" radio button after entering your email address. ****************************************************** Publishers: fantomaster.com Editor-in-Chief: Ralph Tegtmeier M.A. (rt) Technical Editor: Dr. D. Brockhausen (bro) < http://brocon.com > Marketing Editor: Lee Traupel (wbi) < http://intelective.com > Editors: G. Jentzsch (gj), M. Preuss (mp), A. Schmid (as), A. Walker (aw) < mailto:fantomNews@fantomaster.com > URL:< http://www.fantomaster.com > ------------------------ List of Guest Columnists ------------------------ Gabe Weinberg (vol. 1/issue 001) Linda Cox (vol. 1/issue 002+ vol. 2/issue 011) Robert Woodhead (vol. 1/issues 003+004) Lee Traupel (vol. 1/issue 005) Robin Nobles (vol. 2/issues 009+010) Phil Stark (vol. 2/issue 010) David Dimas (vol. 2/issue 012) David Gikandi (vol. 2/issue 012) Eric Lander (vol. 3/issue 013) Mike Grehan (vol. 3/issue 014+015+016) Michael Wong (vol. 3/issue 017) Andrew Gerhart (vol. 4/issue 018) Ammon Johns (vol. 4/issue 018) ****************************************************** The ultimate stealth - the full control Software and services webmasters have been dreaming of for years! Top search engine positioning, automated key word density generation, search bot reference list, killer admin applications, and more: < http://www.fantomaster.com > ****************************************************** (c) copyright 2002 by fantomaster.com All rights reserved. ======================================================