Yahoo!'s Term Extraction Service can be used to extract significant words or phrases from some larger body of text. There are many uses for it, not the least of which is providing keywords, or tags in Web2.0 jargon, to help classify and organize a library of content. The following PHP script uses will use the Term Extraction service to analyze a PDF file. With a little more work, it could be expanded to work with Microsoft Word, Excel, and Powerpoint files. Extracting keywords automatically would be a helpful feature to build into your blog or CMS. There are modules to extract keywords for Drupal and Wordpress.

   1:<?php
   2:// discover where pdftotext tool is
   3:$catpdf = trim(`which pdftotext`);
   4:
   5:// the PDF file to analyze
   6:$source = 'http://example.com/my_file.pdf';
   7:
   8:// will copy file to a local temporary file
   9:$temp_pdf_file = tempnam(sys_get_temp_dir(), "ek");
  10:
  11:// see below
  12:download_file($source, $temp_pdf_file);
  13:
  14:// save text contents of pdf source to another temp file
  15:$extract_file = tempnam(sys_get_temp_dir(), "ek");
  16:exec($catpdf . ' ' . escapeshellarg($temp_pdf_file) . ' ' . escapeshellarg($extract_file));
  17:
  18:// fetch and output terms
  19:$contents = file_get_contents($extract_file);
  20:if ($terms = get_yahoo_terms($contents))
  21:{
  22:    echo "\nYahoo terms for the file $source";
  23:    foreach ($terms as $term)
  24:    {
  25:        echo "\n$term";
  26:    }
  27:    echo "\n";
  28:}
  29:
  30:// hide our footsteps
  31:unlink($temp_pdf_file);
  32:unlink($extract_file);
  33:
  34:/**
  35: * Uses curl to copy $source to a local file $dest
  36: * @param string
  37: * @param string
  38: */
  39:function download_file($source, $dest)
  40:{
  41:    $out = fopen($dest, 'wb');
  42:
  43:    $ch = curl_init();
  44:
  45:    curl_setopt($ch, CURLOPT_FILE, $out);
  46:    curl_setopt($ch, CURLOPT_HEADER, 0);
  47:    curl_setopt($ch, CURLOPT_URL, $source);
  48:
  49:    curl_exec($ch);
  50:
  51:    curl_close($ch);
  52:}
  53:
  54:/**
  55: * Uses curl to query yahoo term extraction service for meaninful terms
  56: * @param string
  57: * @return mixed, array on success or null on failure
  58: */
  59:function get_yahoo_terms($content)
  60:{
  61:    $SERVICE_URL = 'http://api.search.yahoo.com/ContentAnalysisService/V1/termExtraction';
  62:    $app_id = 'F1_Testing';
  63:
  64:    $ch = curl_init();
  65:    curl_setopt($ch, CURLOPT_URL, $SERVICE_URL);
  66:    curl_setopt($ch, CURLOPT_POST, 3);
  67:    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  68:
  69:    curl_setopt( $ch, CURLOPT_POSTFIELDS, 'appid=' . $app_id . '&context=' . urlencode($content) . '&output=php');
  70:    $raw = curl_exec($ch);
  71:    curl_close($ch);
  72:
  73:    if ($raw = unserialize($raw))
  74:    {
  75:        if (isset($raw['ResultSet']['Result']))
  76:        {
  77:            return $raw['ResultSet']['Result'];
  78:        }
  79:    }
  80:}
  81:?>

As a sample of what to expect, I used the script to look at Calculating CARMA: Global Estimation of CO2 Emissions from the Power Sector - Working Paper 145 and the list of terms returned is below. The list of words is fairly accurate, and even includes the name of one of the authors.

global estimation
geographical scales
carbon emissions
co2 emissions
global citizens
global poverty
david wheeler
rigorous research
power plants
power sector
fossil energy
poverty and inequality
solar wind
energy sources
monitoring system
keystrokes
groundwork
strengths and weaknesses
carbon dioxide
aggregation

Toby Ward makes a compelling argument to not worry about the graphic design of your intranet. Often, the first solid, easy to grasp version of any website will be the graphic design. Its the first one people really get, even if you've led them through paper protoypes, wireframes, or very rough html mock ups. But its dangerous and counter-productive to make design high priority because on an intranet, users are very focused on performing a task, like finding a document or filling out an expense report. Too many images, video, bells and whistles will both distract and slow the downloading and rendering of the page. It all comes back to knowing your site's audience, understanding why they come to the site, and building the site to suit them. Read Toby Ward's article for a complete list of the issues that should be considered.

That’s not to say that design (look-and-feel) doesn’t play a roll and isn’t important to users. Design is important, but it doesn’t crack the top 6 or 7 priorities. On average, based on my experience working with dozens of intranet clients, design is equivalent to between 8 – 12% of the total intranet’s value. What is really important is content (20-30%), search (15-20%), information architecture (20-30%), and governance and planning (20-30%).