I’m not going to focus on any of the underhanded tricks to fool search engines into listing an inappropriate page. Google hates this kind of tactic and so do I. You get visits for having good content. Good search rankings are based on good content and the best way to boost your content is to present your page in a way that lets both humans and search engines find the content they want. Part one is on website analysis and tracking.

Don’t assume that your website is easy to navigate. One of the neatest things about the web is that we have instant feedback on use. With just a little tracking we can see what pages people are looking at, how they’re finding our site and what they’re doing once they’re here.

Javascript Trackers:

My most advanced sites are currently using four different tracking methods. Most utilize javascript “bugs,” tiny snippets of code that send individual results to an advanced software tracking system. I put the code inside a Moveable Type “Modules Template” which is automatically imported to all pages. Installing a new system is as easy as cutting-and-pasting the javascript into the Template and rebuilding the site.

• AXS Visitors Tracking System
  This software installs on your server but don’t let that scare you: this is one of the easiest installations I’ve ever seen. AXS gives you great charts of usage: you can narrow it specific pages on your site, or even particular search engines or search phrases.
  There’s also a option to view the lastest traffic by visitor. I love watching this! You can see how individuals are using the site and where they’re navigating. I’ve been able to identify different types of visitors this way and understand the complexity of the audience.
  It doesn’t seem like AXS is not being developed anymore. The latest stable version came out over two years go, which is a shame.

• HitTail
  This service watches search-engine links and makes recommendations for new keywords. I wrote about this service yesterday in Blogging for the Long Tail.

• Reeferss.com
  This is a simple simple bit of software. Like every other tracking system it keeps track of referrers: search engines and websites that bring traffic to your site. But unlike the others that’s all it does. Why care then? It provides a real-time RSS feed of these visitors. I bring the feed into my “Netvibes” page (a customized start page, see below) and scan the results multiple times a day.

• Google Analytics
  The internet’s gatekeeper bought the Urchin analytics company in April 2005 and relaunched the product as Google Analytics shortly thereafter. This is becoming an essential tracker. It’s free and it’s powerful, though I haven’t been as impressed by it as others have. See its Wiki page for more.

Internet Trackers:

It’s easy to find out what people are saying about you online.

• Technorati
  This service tracks blogs but you don’t need to have a blog to use it, for Technorati will tell you where blogs are linking. Give it your URLs (or those of your competitors!) and you’ll know whenever a blogger puts in a link to you. You can also give it keywords and find out when a blog uses them.

• Google Blog Search
  Google can also let you follow blog references or keyword mentions on the blogs. Google will also track beyond blogs of course. Type “site:www.yourdomain.com” into the main Google search page and you’ll see who’s linking to your site (or to the competition). There are lots of other services that track blogs and mentions–Sphere, Bloglines, etc. They all have different strengths so try them and see what you think.

• Feedburner
  The best RSS massager has always focused on ways to track your RSS feed. They’ve recently introduced page tracking software too. It looks great but I just installed it this week. I still have to see if it’s as good as Feedburner’s other offerings.

Keeping on top of this flow of data:

It’s easy to get overwhelmed by all of this information. Most of the tracking services provide RSS feeds (See The Wonders of RSS Feeds for an intro). I use Netvibes, a customized start page, to pull these all together into a single page that I can scan every morning. Here’s a screenshot of part of my Netvibes tracking page–the full page currently shows fourteen tracking feeds on one screen:

So why is tracking important to SEO?

With tracking you find out what people are looking for on the internet. This helps you create pages and services that people will want to find. You might be surprised to see what they’re already finding on your site. Some examples:

• Analyzing one site, I noticed that few pages I thought were obscure were bringing in high Google traffic. I looked at these pages again and realized they did a good job of describing the company’s mission. I consequently redesigned the site homepage to feature them and I made sure that those pages contained direct links to its most important services.

• When I started work for another client I looked at their site and suspected that they’re most important articles were not being seen–visitors had to click through about four times to get to them. Six months of tracking confirmed my hunch and gave me the hard data to convince the executive director that we made some small modifications to the design. Having this strong content linked right off the homepage helped bring in Google traffic.

A lot of the search phrases are predictable for anyone who reads my blog enough: “modern liberal Quakers”:google, “Quaker peace testimony”:google, “Quaker decline”:google, “Quaker theology”:google, “emergent church movement”:google and “catholic Quakers”:google. By far the most popular searches are for the plain dress page. Every day I get searches for “modest dresses”:google, “plain dress”:google, “Quaker dress”:google, etc. Most Quakers might have long ago dismissed peculiarities like plain dress as relic of the nineteenth century, but a lot of twenty-first century net surfers are curious about this tradition of ours.
Sometimes I get search traffic that is downright bizarre. Who searches for “Its light that makes me uncomfortable?”:google (I like it; the Light spoken of by Friends is one that exposes and convicts before it comforts). “I’m going to hire a wino to decorate our home”:google is not a tactic I’ve ever considered (thanks Melynda). I’m apparently a world expert in “insecurities of young people from fashion modeling.”:google And if you want to know if “armageddon [is] gods way of getting rid of human race”:google I’m the guy to talk to. My Lutheran grandmother will rest easier in her grave knowing that I’m an important figure for “christian young adults”:google and a leading voice on “morality in twenty somethings”:google, but if all this righteousness gets to you I can also show you how to “beat a dead horse”:google.
More in the bizarro “why me?!” file: “baby arm picture”:google, “hand wash experiments”:google, “liberal protestantism and safe sex”:google, “unused cell phone numbers”:google. I’m not sure who thinks I know anything about the “statue of liberty holding a guitar”:google. “Do amish women wear bras”:google?: I don’t know.
Some of the phrases are so generic that I marvel that they point here. Are there really so few sites talking about “twenty-somethings”:google, even generically? Shouldn’t there be lots of mainline Protestants worried about their declining numbers and asking “why are churches dying”:google. There ain’t much movement to the “emergent church movement”:google if I’m the number one hit. I’d be happy to guide visiors to “gay christian websites”:google but I’m hardly an expert (or does Google know something about me that I don’t?). I’ve never been asked to give any major “Quaker speeches for peace”:google even though Google seems to think it’s about time; if you want lighter fare for your conference, I can also give a presentation on “fun things to to do with your Quaker”:google.
For the record: I have never met “mel gibson’s wife”:google, though I do know “Theo‘s mom”:google quite intimately, having been “married in south jersey”:google (want proof? How about some “baby Quaker pictures”:google ?). I don’t run “the social network for gorillas”:google but if I did it stands to reason I’d be something of an authority on the “theology of the planet of the apes”:google. If I knew “where thriving young adults can be successful”:google, do you think I’d be working for nonprofits?? I’m also afraid I don’t have much advice on “how to flatten new sod”:google. I do agree that “there were no good old days, these are the good old days”:google.
Finally, my favorite search phrase: “baby theo”:google. I have at least one Friend that uses this search phrase instead of bookmarking my site (he complained when my Baby Theo page temporarily fell out of first place).

h4. Methodology
bq. The linked words in this post are a sample of actual phrases that have brought actual visitors to my site from search engines. All of the links are to Google, the most commonly-used search engine, but some of these visitors used other search engines to find my site (which is why I won’t necessarily come up when you click the Google link).
h4. Updates
* “How crazy am I survey”“http://www.google.com/search?hl=en&lr=&safe=active&q=HOW%20CRAZY%20AM%20I%20SURVEY (9/2006)