Developer, Entrepreneur, American

I've been doing some SEO stuff for <a href="http://www.greatdentalwebsites.com">Great Dental Websites</a> and I had a need to automatically generate some keywords and meta data.

In order to accomplish my particular task, I need a histogram of all the words in a long blob of text.  I was shocked to not be able to find anything on this in written in CF, so I set out to write my own:

<cfset faqText = getAllFAQs.question & " " & stripHTML(getAllFAQs.answer) />
<cfset skipwords = "all,another,any,anybody,anyone,anything,both,each,either,everybody,everyone,everything,few,he,her,hers,herself,him,himself,his,I,it,its,itself,little,many,me,mine,more,most,much,myself,neither,no,one,nobody,none,nothing,one,one another,other,others,ours,ourselves,several,she,some,somebody,someone,something,that,theirs,them,themselves,these,they,this,those,us,we,what,whatever,which,whichever,who,whoever,whom,whomever,whose,you,yours,yourself,yourselves,,a,the,to,are,of,can,is,but,have,that,want,What,my,an,for,all,out,and,look,very,need,get,case" />
------
<cfoutput>#getHistogram(faqText,skipwords, 10)#</cfoutput>

<cffunction name="getHistogram" returntype="array" hint="Creats a histogram of words">
    <cfargument name="sourceText" required="true" hint="The string of text we want to generate a histogram for" type="string" />
    <cfargument name="ignoreList" required="false" hint="comma delineated list of words to ignore" type="string" />
    <cfargument name="histogramLength" required="false" hint="number of words that we want to send back..ie only the top 5" type="string" />

    <cfset var histogramCount = structNew() /> <!--- our histogram! --->
    <cfset var sortedHistogram = "" />  <!--- a sorted array of our histogram --->
    <cfset var x = "" /> <!--- iterator --->
    <cfset var i = "" /> <!--- iterator --->
   
    <!--- loop through all of the text, assuming that a space separates a word --->
    <cfloop delimiters=" " list="#sourceText#" index="i">
   
        <!--- see if we have this already in our struct --->
        <cfif structKeyExists(histogramCount, "#i#")>
            <!--- we do! increase its count by 1 --->
            <cfset histogramCount[i] = histogramCount[i] + 1 />
        <cfelse>
            <!--- we do not, make a new key in the struct for this word --->
            <cfset histogramCount[i] = 1 />
        </cfif>
    </cfloop>
   
   
    <!--- Do we have an ignore list? --->
    <cfif structKeyExists(arguments, "ignoreList") and len(trim(arguments.ignoreList))>
        <!--- loop over the list of ignore words and remove any matches from our structure --->
        <cfloop delimiters="," list="#arguments.ignoreList#" index="x">
            <!--- does this word occur in our struct? --->
            <cfif structKeyExists(histogramCount, x)>
                <!--- yes, so remove it --->
                <cfset structDelete(histogramCount, x) />
            </cfif>
        </cfloop>
    </cfif>

    <!--- Sort the histogram based on most occurences of a given word --->
    <cfset sortedHistogram = StructSort(histogramCount, "numeric", "DESC") />

    <!--- see if we need to only show x number of words for this histogram --->
    <cfif structKeyExists(arguments, "histogramLength") and len(trim(arguments.histogramLength))>
        <cfset useNum = arguments.histogramLength + 1 />
        <cfloop index="y" from="#arrayLen(sortedHistogram)#" to="#useNum#" step="-1">

            <cfset ArrayDeleteAt(sortedHistogram, y) />       
        </cfloop>
    </cfif>

    <cfreturn sortedHistogram>

</cffunction>

One thing I would like to improve is for my function to return not only the list of words in terms of how often they came up, but also the number of times they came up.  This data gets dropped when we sort the struct into an array (which is a weird).

Any suggestions?

I frequently have to switch between two development environments.  One is an Adobe CF environment, and another is a Railo environment.  Both need unique apache httdp.conf, so i wrote two batch files to take care of this.  Then I just run them from within eclipse, and restart apache!

Here are the files:

switch_cf.bat

:: Switch to coldfusion
If exist cf_httpd.conf (
rename httpd.conf r_httpd.conf;
rename cf_httpd.conf httpd.conf;
)

switch_railo.bat

:: Switch to railo
If exist r_httpd.conf (
rename httpd.conf cf_httpd.conf;
rename r_httpd.conf httpd.conf;
)

Pretty simple!

By now you've all seen the season finale to Battlestar Galactica, which was quite an awesome show.  However, the ending left me feeling a bit unfulfilled, and some parts were completely unplausible: I just can't imagine that everyone was willing to just give up all their technology and live a life of toil and struggle amongst the sabre tooth tigers for the rest of their lives after 4 years of hardship, pain, and struggle.

Anyway
The first part of the finale was just great, right upto the point where they jumped away from the colony, which is where we pick up...

They jump away from the cylon colony and into earths orbit, 12 hours later the fleet shows up.  Helo, who is in critical condition after being shot, is being moved on a raptor to the medical ship with doc hoddle, who is gearing up to prep him for emergency surgery.  Hell, I'll even put apollo and starbuck on the raptor flying it for excitement purposes.  Meanwhile around the fleet signs of jubilation occur as everyone discovers they have found one sweet ass home. Unfortunately, galactica's battle damage is too much, and the ship begins to go critical.  Adama orders an evacuation but its too late and the ship goes nuclear, destroying the entire fleet in the process.  The raptor with helo and his family on it is the only military aircraft far enough away (we'll put the medical ship at the farthest point away from galactica for simplicity sake) that is able to survive the galactica, all the other ships are unshielded and blow the fuck up.  The raptor is damaged though and has to crash land on earth, and its angling towards africa.

Because of the blast, the baseship has no idea that helo and hira and sharon survived because all their sensors are fucked up, and they fail to detect the raptor entering earths atmosphere.  Eventually The baseship decides to leave and jumps away after the centurions decide that all the humans must be dead.

We spend about 2 minutes with the people on the raptor as it goes down and they all conteplate their impending death.  Miraculously, starbuck and appollo manage to crash land the raptor.  As they land they hit a a rock and part of it crushes the cockpit, instantly killing starbuck and leaving apollo fatally injured, dying in his chair.  sharon drags Helo, out of the cockpit hira scrambles out as well, but the raptor explodes, wounding sharon and hira as well.  Now the native tribe show up and attempt to help helo and sharon but its too late and they die.  Hira fairs better, and is adopted by the tribe..

Fast forward 20 years to the final scene.  Hira is believed to be a god by the native tribe, and has produced countless offspring.   She is the new queen. The tribe that has adopted her has adopted language and is making huge strides towards civilization.  Signs of other tribes in slavery are throughout the village  As hira overlooks her troops preparing to enslave yet another rival tribe, we zoom in on her eyes, they flash red from left to right and back again.  She is still, and will always be part cylon.