Introduction

One of the SharePoint data sources are HTML Web pages.

SharePoint crawler retrieves metadata from Web content as crawled properties.

Default crawled properties available before you perform a first crawl are listed here^[1].

List of crawled properties as well as list of crawled property categories will change after the first crawl to include additional crawled properties and categories found during that crawl.

One example of custom crawled properties Sharepoint is retrieving from Web pages are  HTML meta tags content.

1             Overview

The crawl properties that the SharePoint search crawler creates for the custom HTML meta tags are rendered to following crawled properties categories: 

• Web
• Document Parser

Web is standard crawled properties category. Crawled properties are added to this category with the same name as the name of the custom meta tags but made uppercase.

Document Parser is custom crawled properties category. Crawled properties are added to this category with the same name as the name of the custom meta tags and in the same case.

 Example:

Web page http://authors.library.caltech.edu/37214/ contains meta tag -

 <meta name="eprints.citation" content="  DiMarco, E. Joseph and Khabiboulline, Emil and Orris, Darryl F. and Tartaglia, Michael A. and Terechkine, Iouri  (2013) Superconducting Solenoid Lens for a High Energy Part of a Proton Linac Front End.  IEEE Transactions on Applied Superconductivity, 23  (3).   Art. No. 4100905.  ISSN 1051-8223     http://resolver.caltech.edu/CaltechAUTHORS:20130228-145009767 &lt;http://resolver.caltech.edu/CaltechAUTHORS:20130228-145009767&gt;  " />

 After crawling this page SharePoint crawler will create following crawled properties:

Web category -

 Property name: EPRINTS.CITATION
 Category: Web
 Property Set ID: d1b5d3f0-c0b3-11cf-9a92-00a0c908dbf1 

Document Parser category -

 Property name: eprints.citation
 Category: Document Parser
 Property Set ID: 64ae120f-487d-445a-8d5a-5258f99cb970

2             HTML meta tags and corresponding crawled properties

There are many meta tags namespaces used by different publishers.

The most common are:

              Highwire Press tags (e.g., citation_title),

              Eprints tags (e.g.,eprints.title),

              Dublin Core tags (e.g., DC.title)

2.1              Crawled properties for Citation Meta tags

2.2              Crawled properties for Eprints Meta tags

2.3              Crawled properties for Dublin Core Meta tags