ThingWorx Model Definition and Composer > Things > Content Crawler > Content Crawler (Legacy Composer)
  
Content Crawler (Legacy Composer)
* 
For information about a Content Crawler Thing and for steps to create one in new Composer, see Content Crawler.
A Content Crawler Thing is used to call a service on another entity. It is used to retrieve data and store the data in the data table of the Content Crawler Thing.
On a separate entity from the Content Crawler Thing, a service must be defined that fetches data and returns an INFOTABLE of that data back to the Content Crawler. The Content Crawler then maps the incoming fields as well as the tags to the fields used in the Data Shape for the Content Crawler. Each row is added as a new entry to the Data Table on the Content Crawler Thing. The index of the Content Crawler’s Data Table functions in the same way as a normal Data Table entity.
Creating a Content Crawler
To retrieve data from the data table of an entity into the data table of a Content Crawler Thing, do the following in legacy Composer.
1. Create a data shape and define fields to use in a data table.
2. Create a data table with the data shape created in the previous step.
3. Create a custom service on the data table that includes an output of the INFOTABLE base type and the same data shape used in the previous step.
4. Set the Infotable Type to Is Content Crawler Entry.
5. Create a new data shape for the Content Crawler thing.
* 
You can create a new Content Crawler-specific data shape, or you can use the same data shape that was used in the data table created in step 1. Although this step is optional, we will use a new data shape for the Content Crawler Thing in this example.
6. Create a new Content Crawler thing:
a. Create a thing.
b. In the Thing Template field, select Content Crawler.
c. In the Data Shape field, select the data shape you created in the previous step.
Content Crawler Configuration
The Configuration page for the Content Crawler Thing contains configuration tables that allow you to map fields from the retrieved data.
The Field to Field Mappings configuration table maps the fields from the retrieved data to the fields defined on the data shape of the Content Crawler Thing.
* 
If the same data shape is used on the Content Crawler thing and for the infotable returned from the Content Crawler service, field mappings are not necessary as they are handled automatically.
The Field to Tag Mappings configuration table maps the values of a field to tags in a data tag vocabulary.
When the data tag vocabulary is dynamic, any value being mapped from the data will have a term automatically entered into the vocabulary.
When the data tag vocabulary is not dynamic, any value being mapped from the data will need a predefined term representing the value to be mapped properly.
For example: TestingVocab:false;TestingVocab:iAmAString. The first part is the boolProp value and the second part is the stringProp value.
Content Crawler Services
The following services are unique to the Content Crawler thing:
CrawlEntries: purges all of the data table entries for the Content Crawler and then executes GetExternalContent.
GetExternalContent: executes the service defined on the General Information page of the Content Crawler Thing. An INFOTABLE of retrieved values is returned from the service. No modifications to the data table of the Content Crawler are performed.
GetExternalContentDetail: retrieves a specific content item by key.