Use Google’s Power To Create Powerfull Search Engines (Part II)

In our previous article, we learned how to create a custom search engine using Google’s infrastructure. In this article, we will go through some remaining aspects of this wonderful API and we will also create a custom search engine using what we learned till now.

Refinements

Refinements AKA search suggestions are little tips that we show to our users to help them narrow their search. For example, if someone searches for “python”, our search engine does not know if the user is looking for the snake family of pythons or the programming language. So, we could help our users refine their search by providing some search suggestions like:

• python language
• python programming
• python tutorial
• python snake
• etc

To create refinements you need to either add them through the control panel or by using an annotations XML file. The code needed to add each refinement label is:

<Facet>
 <FacetItem title="Lectures">
 <Label name="lectures" mode="BOOST" weight="0.8"/></Label>
 </FacetItem>
 </Facet>

The Facet element has this structure:

• Facet
  • FacetItem - The refinement link that is displayed at the top of the results page.
    • Label name
      • Rewrite
      • IgnoreBackgroundLabels
    • Redirect

Each Facet elements can have up to four FacetItem child elements. You can create as many refinement labels as you want and display up to 16 refinement links in the search results page. Here is an example on how we can add the above example to a custom search engine file:

<CustomSearchEngine>
 <Title>Universities</Title>

 <Context>

 <Facet>
 <FacetItem title="Lectures">
 <Label name="lectures" mode="BOOST" weight="0.8">
 <Rewrite>lecture OR lectures</Rewrite>
 </Label>
 </FacetItem>
 </Facet>

 <Facet>
 <FacetItem title="Assignments">
 <Label name="assignments" mode="BOOST" weight="0.8">
 <Rewrite>homework OR assignment OR assignments</Rewrite>
 </Label>
 </FacetItem>
 </Facet>

 <BackgroundLabels>
 <Label name="_cse_omuauf_lfve" mode="FILTER"/>
 <Label name="_cse_exclude_omuauf_lfve" mode="ELIMINATE"/>
 </BackgroundLabels>

 </Context>

 </CustomSearchEngine>

Tagging Sites With Refinement Labels

You can tag sites with refinement labels. Each annotation can have multiple labels, which means that the same site can be used in other search engines and be ranked differently. This example shows how to do this in the annotations file:

<Annotations>
 <Annotation about="webcast.berkeley.edu/*" score="1">
 <Label name="university_boost_highest"/>
 <Label name="lectures"/>
 </Annotation>

 <Annotation about="www.youtube.com/ucberkeley/*" score="1">
 <Label name="university_boost_highest"/>
 <Label name="videos_boost_mid"/>
 <Label name="lectures"/>
 </Annotation>
</Annotations>

Appending Search Queries

If you want to help your users by providing refinement links that add helpful search terms to their queries, you can use the Rewrite element in your refinements. The element appends the search terms to your users’ queries when they click a refinement link. The Rewrite element can have up to 100 characters, all of which should be lowercase, except for uppercase search operators such as OR. So based on the refinement labels we can use the rewrite child element and do this:

<Facet>
 <FacetItem title="Homework">
 <Label name="assignments" mode="BOOST">
 <Rewrite>homework OR assignment</Rewrite>
 </Label>
 </FacetItem>
 </Facet>

Now. Each time the user clicks on the Homework refinement link, the query will be appended with “homework OR assignment”. This way, our search engine will show more relevant results  than it would for just homework.

Redirecting Search Queries

You can also redirect the user to another site when he/she clicks on a refinement label. For example you can redirect a user to the PHP Manual when he clicks on the refinement link. This can be done like this:

<CustomSearchEngine>
 <Title>Universities</Title>
 <Context>
 <Facet>
 <FacetItem title="Manual">
 <Label name="manual" mode="FILTER"/>
 <Redirect url="http://php.net/manual/en/$q"/>
 </FacetItem>
 </Facet>
 </Context>
</CustomSearchEngine>

Using Synonyms

You can expand your users’ search queries by using synonyms, which are variants of a search term. This helps the users since when a user searches for “photoes” our search engine will bring up results for “images”, “landscapes” and “screenshots” too. While the example mentioned is many times handled by Google itself, some other words might not. So for example a user that searches for “sf” might sometimes looking for Symfony, others for Sourceforge and some other times for San Francisco. It is up to you to determine what you want the engine to bring up based on your audience. To add synonyms you must add the code in the Context file of your CSE.

Example:

<Synonyms>
 <SynonymEntry word="stock">
 <Synonym>equity</Synonym>
 <Synonym>share</Synonym>
 </SynonymEntry>
 </Synonyms>

Each query term (SynonymEntry element) can have with up to ten variants (Synonym elements). The variants could be single words or phrases. You can define multiple query terms, so long as you not exceed a total of 500 variants.

Creating Special Results

For any reason, you might want to bring a search result that you think that it answers your user’s queries to the top by creating special results. The two types of special results are:

• Promotions - simple results with links to relevant webpages.
• Subscribed links – result snippets for a set of queries. Subscribed Links directly answers your users’ questions and save them from some research work. A type of subscribed link is this one that answers the question to everything.

Before you start creating special results, determine which type best suits your needs. Promotions are simpler and easier to create, while subscribed links have richer features but require significantly more planning, development, and troubleshooting.

Promotions

Unlike other Custom Search features, you create the XML for promotions not in the context file nor the annotations file, but in its own file. The following code describes promotions for a search engine on programming tutorials:

<Promotions>
 <Promotion id="1"
 queries="tutorial, programming tutorial"
 title="Programming Tutorials"
 url="http://jeez.eu/category/tutorials/"
 description="Programming tutorials for PHP, JavaScript, Web Services and more..."
 image_url="http://d.yimg.com/cc/img/resize/SIG=1395nor9l/*/74x74/http%3A%2F%2Fd.yimg.com%2Fgg%2Fjeeztech%2Favatars%2F52141fe00e1281a84745259ab5eec6b46e23c412.png" />
 <Promotion id="2"
 queries="css,css tutorial"
 title="CSS Tutorials"
 url="http://www.smashingmagazine.com/category/css/"
 description="CSS tutorials" />
</Promotions>

You can have up to 2,000 promotions in your account. You can allocate the promotions across the search engines in your account any way you like. Let’s say you have two search engines in your account. You can apportion 2,000 promotions to one search engine and nothing to the other; or you can apportion up to 1,000 promotions to each search engine.

Subscribed Links

To better understand the subscribed links feature, just go to Google and search for “AJAX”. The first result is the “AJAX (programming)” article on wikipedia. If you wanted your users to get info about the AJAX hero, you could use a subscribed link to do this.

To enable the subscribed links you created using the control panel:

1. Go to the Basics tab.
2. Select the Enable special results, such as subscribed links and promotions check box.
3. Click Save Changes.

and to add them in the context file you should do something like this :

<CustomSearchEngine volunteers="false">
 <Title>...</Title>
 <Description>...</Description>
 <Context>...</Context>
 <SubscribedLinks>
 <SubscribedLink creator="017771777217723414381"/>
 </SubscribedLinks>
</CustomSearchEngine>

You can find the SubscribedLink creator id you should download the context file generated by the control panel and find the creator attribute of the CustomSearchEngine element.

Creating a Demo Search Engine

As we promissed, we are going to create a demo search engine that will facilate most of what we learned through these two articles about custom search engines. Try this search engine. Try to search for something that has to do about programming and also try searching for “css”,”css tutorial”,”tutorial” and “programming tutorial”.

These are the files:

Annotations file

<?xml version="1.0" encoding="UTF-8" ?>
<Annotations start="0" num="21" total="21">
 <Annotation about="*.jeez.eu/*" timestamp="0x000476ee0956ea94" href="CgsqLmplZXouZXUvKhCU1dvK4N2dAg">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="jeez.eu" />
 </Annotation>
 <Annotation about="www.scala-lang.org/node/198*" timestamp="0x000476edc86a7b49" href="Chx3d3cuc2NhbGEtbGFuZy5vcmcvbm9kZS8xOTgqEMn2qcPc3Z0C">
 <Label name="manual" />
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.scala-lang.org/node/198" />
 </Annotation>
 <Annotation about="groovy.codehaus.org/Documentation*" timestamp="0x000476edc7508a05" href="CiJncm9vdnkuY29kZWhhdXMub3JnL0RvY3VtZW50YXRpb24qEIWUwrrc3Z0C">
 <Label name="manual" />
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://groovy.codehaus.org/Documentation" />
 </Annotation>
 <Annotation about="guides.rubyonrails.org/*" timestamp="0x000476edc3b646e8" href="ChhndWlkZXMucnVieW9ucmFpbHMub3JnLyoQ6I3ZndzdnQI">
 <Label name="manual" />
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://guides.rubyonrails.org/" />
 </Annotation>
 <Annotation about="java.sun.com/docs/books/tutorial/*" timestamp="0x000476edc11894bc" href="CiJqYXZhLnN1bi5jb20vZG9jcy9ib29rcy90dXRvcmlhbC8qELyp4ojc3Z0C">
 <Label name="manual" />
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://java.sun.com/docs/books/tutorial/" />
 </Annotation>
 <Annotation about="php.net/docs.php" timestamp="0x000476edbfe2de08" href="ChBwaHAubmV0L2RvY3MucGhwEIi8i__b3Z0C">
 <Label name="manual" />
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://php.net/docs.php" />
 </Annotation>
 <Annotation about="*.google.com/*" score="1" timestamp="0x000476afacb2e2c1" href="Cg4qLmdvb2dsZS5jb20vKhDBxcvl-tWdAg">
 <Label name="_cse_cew2vi6nsvu" />
 <AdditionalData attribute="original_url" value="http://google.com" />
 </Annotation>
 <Annotation about="www.ajaxgear.com/*" score="1" timestamp="0x000476af5a53644a" href="ChJ3d3cuYWpheGdlYXIuY29tLyoQysjN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.ajaxgear.com" />
 </Annotation>
 <Annotation about="qooxdoo.oss.schlund.de/*" score="1" timestamp="0x000476af5a536444" href="Chhxb294ZG9vLm9zcy5zY2hsdW5kLmRlLyoQxMjN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://qooxdoo.oss.schlund.de" />
 </Annotation>
 <Annotation about="www.zimbra.com/*" score="1" timestamp="0x000476af5a53643e" href="ChB3d3cuemltYnJhLmNvbS8qEL7IzdL11Z0C">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.zimbra.com" />
 </Annotation>
 <Annotation about="cpaint.booleansystems.com/*" score="1" timestamp="0x000476af5a536437" href="ChtjcGFpbnQuYm9vbGVhbnN5c3RlbXMuY29tLyoQt8jN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://cpaint.booleansystems.com/" />
 </Annotation>
 <Annotation about="www.plextk.org/*" score="1" timestamp="0x000476af5a536432" href="ChB3d3cucGxleHRrLm9yZy8qELLIzdL11Z0C">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.plextk.org/" />
 </Annotation>
 <Annotation about="script.aculo.us/*" score="1" timestamp="0x000476af5a53642b" href="ChFzY3JpcHQuYWN1bG8udXMvKhCryM3S9dWdAg">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://script.aculo.us/" />
 </Annotation>
 <Annotation about="www.bosrup.com/web/overlib/*" score="1" timestamp="0x000476af5a536424" href="Chx3d3cuYm9zcnVwLmNvbS93ZWIvb3ZlcmxpYi8qEKTIzdL11Z0C">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.bosrup.com/web/overlib/" />
 </Annotation>
 <Annotation about="moofx.mad4milk.net/*" score="1" timestamp="0x000476af5a53641d" href="ChRtb29meC5tYWQ0bWlsay5uZXQvKhCdyM3S9dWdAg">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://moofx.mad4milk.net/" />
 </Annotation>
 <Annotation about="bennolan.com/behaviour/*" score="1" timestamp="0x000476af5a536416" href="ChhiZW5ub2xhbi5jb20vYmVoYXZpb3VyLyoQlsjN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://bennolan.com/behaviour/" />
 </Annotation>
 <Annotation about="developer.berlios.de/projects/bajax/*" score="1" timestamp="0x000476af5a53640d" href="CiVkZXZlbG9wZXIuYmVybGlvcy5kZS9wcm9qZWN0cy9iYWpheC8qEI3IzdL11Z0C">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="https://developer.berlios.de/projects/bajax/" />
 </Annotation>
 <Annotation about="www.dojotoolkit.org/*" score="1" timestamp="0x000476af5a536406" href="ChV3d3cuZG9qb3Rvb2xraXQub3JnLyoQhsjN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.dojotoolkit.org/" />
 </Annotation>
 <Annotation about="www.mochikit.com/*" score="1" timestamp="0x000476af5a536400" href="ChJ3d3cubW9jaGlraXQuY29tLyoQgMjN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://www.mochikit.com/" />
 </Annotation>
 <Annotation about="openrico.org/rico/home.page" score="1" timestamp="0x000476af5a5363f9" href="ChtvcGVucmljby5vcmcvcmljby9ob21lLnBhZ2UQ-cfN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://openrico.org/rico/home.page" />
 </Annotation>
 <Annotation about="prototype.conio.net/*" score="1" timestamp="0x000476af5a5363eb" href="ChVwcm90b3R5cGUuY29uaW8ubmV0LyoQ68fN0vXVnQI">
 <Label name="_cse_ytl_pcjhu6w" />
 <AdditionalData attribute="original_url" value="http://prototype.conio.net" />
 </Annotation>
</Annotations>

Context File

<?xml version="1.0" encoding="UTF-8" ?>
<CustomSearchEngine id="ytl_pcjhu6w" creator="014105238919074316991" keywords="php programming &quot;web development&quot;" language="en" encoding="UTF-8">
 <Title>JSLSE</Title>
 <Description>JavaScript Libraries Search Engine</Description>
 <Context>
 <Facet>
 <FacetItem>
 <Label name="manual" mode="BOOST">
 <Rewrite>&quot;php manual&quot; OR &quot;java manual&quot; OR &quot;ruby manual&quot; OR &quot;groovy manual&quot; OR &quot;scala manual&quot; OR &quot;python manual&quot;</Rewrite>
 </Label>
 <Title>Manual</Title>
 </FacetItem>
 </Facet>
 <BackgroundLabels>
 <Label name="_cse_ytl_pcjhu6w" mode="BOOST" />
 <Label name="_cse_exclude_ytl_pcjhu6w" mode="ELIMINATE" />
 </BackgroundLabels>
 </Context>
 <LookAndFeel>
 <Promotions show_image="true" show_snippet="true" />
 </LookAndFeel>
 <SubscribedLinks>
 <SubscribedLink creator="014105238919074316991" />
 </SubscribedLinks>
 <AdSense />
 <EnterpriseAccount />
</CustomSearchEngine>

Popularity: 1%

Related posts:

1. Use Google’s Power To Create Powerfull Search Engines (Part I) Did you know that you can use Google’s power to...
2. Using Bing’s API To Create A Custom Search Engine Microsoft’s search engine Bing, is a getting more popular each...
3. Google Visualizations From A To Z Google has released an API that you can use to...
4. Google Maps From A to Z During the last years, online maps have evolved so much...
5. Bing Maps. A Google Maps Alternative, or Better? Microsoft has released their own maps service and to be...