Specialized Administration > Configuring Your Windchill Environment > Windchill Search Administration > Index Search Administration > Managing the Search Index > Bulk Index Tool
  
Bulk Index Tool
When indexing is enabled and operating correctly, an indexable object is indexed as soon as it is created. However, there are times when you need to index large amounts of data at one time.
You can use the Bulk Index Tool to load Windchill Index Search libraries and their objects:
• Build indexes of existing data that belong in an index according to index policy.
• Start and stop the bulk indexing process.
• Schedule the process to start and stop at specified times.
• Check the status of the overall bulk indexing process.
• Attempt to re-index objects that have failed the indexing process.
• Maintain a detailed log of the indexing process.
• Reinitialize a Windchill Index Search library after changes have been made to the indexing policies.
• Reinitialize a Windchill Index Search library after changes have been made to the publishing policies. This includes changes made to the com.ptc.windchill.partslink.publisher.publishAll property.
Because loading indexes can take a significant amount of time, it might be necessary to stop the operation for some length of time. The index state is maintained in the IndexStatus table. This allows you to stop and restart the bulk indexing process without re-indexing objects.
The basic process of the Bulk Index Tool is as follows:
1. Query Windchill for all applicable objects.
2. Compare those objects to the IndexStatus table to determine if they have been indexed.
3. Determine whether each object belongs in a collection according to the index policy of the domain to which the object belongs.
4. If applicable, the object is indexed into the appropriate collection.
* 
Windchill only indexes the latest iteration of any revision.
If a user searches for the latest iteration of an object that was loaded using the data loading utilities, all iterations appear in the search results. You can correct this problem by using the Bulk Index Tool to re-index the data once it has been loaded.
Prerequisites and Performance Improvement Tips
• To improve performance, disable indexing when bulk loading data. Once the bulk loading operation is complete, re-enable the Bulk Index Tool and populate your indexes with the new data.
• If you plan to modify the default MIME file types for content indexing, follow the procedure outlined in Specify MIME Types for Content Indexing prior to running the Bulk Index Tool.
• If you are indexing large amounts of data comprised of content holder objects (such as documents), consider setting the wt.index.BulkIndexContentQueueList property. This property establishes separate indexing queues for content holder objects. For more information, see Windchill Search Properties.
* 
If bulk indexing is already running on the server, then before setting this property you must delete and recreate the existing indexing queue entries. To do this, select option 2 from the Bulk Index Tool to stop bulk indexing. After some time has passed, use the Queue Management utility to verify that all indexing queue entries have been cleared.
Solr Commit Frequency
To reduce the frequency of Solr commits, increase the amount of time between auto-commit events:
1. Navigate to the following file:
<Index_Search_Home>\SolrServer\solr\solr-home\wblib\conf\solrconfig.xml
2. Increase the maxTime value from 60000 ms to 300000 ms:
<autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>300000</maxTime>
</autoCommit>
3. Restart the server and start indexing using the Bulk Index Tool.
4. When bulk indexing is complete, revert the maxTime value back to 60000 milliseconds:
<autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>60000</maxTime>
</autoCommit>
For more information, see autoCommit.
Using the Bulk Index Tool
To use the Bulk Index Tool, you must log in as an administrator.
Open a Windchill shell and enter the following command:
windchill wt.index.BulkIndexTool
If you are performing bulk indexing on a server that has both Windchill Index Search and Windchill PartsLink installed, the Bulk Index Tool prompts you to select a core library (collection). The default library is wblib.
* 
You can open two command prompts, side by side, to simplify the process of running the tool. Use one command prompt to run the Bulk Index Tool and the other command prompt to tail the BulkIndexTool.log file.
The tail utility is a standard UNIX utility. This utility is also available for Windows from GNU at the following website:
For real-time progress, you can run the tail utility on the BulkIndexTool.log file. For example:
tail -f BulkIndexTool.log



Bulk Index Tool Menu Options
0
View the current value of the following properties:
• wt.index.enabled
• com.ptc.windchill.partslink.index.enabled
• wt.index.bulkIndexSize
• wt.index.BulkIndexQueueList
• wt.index.BulkIndexContentQueueList
• wt.index.maxContentSize
• wt.index.checkDocumentTypes
• wt.index.filterFileTypes
• wt.index.IndexAndSearchMetaDataOnly
• wt.index.checkIndexingRulesBeforeQueue
• wt.index.excludeAttributes
* 
This option is only available when you have selected a core eligible for federated library searches.
1
Start the bulk indexing process
Select this option to begin indexing your data. This option also creates an entry in the BulkIndexQueue, which executes the actual bulk indexing task.
If you have previously started the bulk indexing process and it is still running when you select this option, you receive an error message.
2
Stop the bulk indexing process.
Select this option to stop the bulk index loading process and remove any remaining bulk indexing queue entries.
* 
If you do not select this option, then you should monitor the queue to ensure that multiple entries are not generated.
3
Schedule the bulk indexing process.
Select this option to set up a regular schedule to repeat the bulk indexing process. You might want to schedule this time during low user activity.
Enter the following information:
• Start time. Use the format mm/dd/yyyy hh:mm am/pm.
• Stop time. Use the format mm/dd/yyyy hh:mm am/pm.
• Total number of runs (how many times you want the scheduled task repeated).
• Frequency (in days) that you want the bulk indexing task to run. (For example, enter 1 for daily; enter 7 for weekly.)
4
Reset failed entries
Select this option to reset the objects that failed during indexing. You can use this option to process failed objects again.
5
Reset entries that are processing.
Select this option if you have objects that have not yet been marked as complete. This can happen if communication between the indexing engine and Windchill occurred and Windchill did not update the object.
6
Reset entries that have no indexing policies.
Select this option if you have changed indexing rules and objects that were previously not indexed should now be included.
* 
Creating a new indexing policy rule does not affect objects that have already been indexed.
7
Check the bulk indexing progress:
Select this option to view indexing status.
The following status example indicates that out of 3609 objects, 3588 are indexed, 15 objects failed, and 6 objects have yet to be indexed.
Current status of Bulk Index:
Total Objects Handles: 3609
Objects processed: 3588
Objects processing: 0
Objects w/o indexing policies: 0
Objects remaining: 6
Objects failed: 15
When all objects have been processed, the bulk indexing process is complete.
* 
This progress is dependent on the wt.index.bulkIndexSize property. No changes to status are made until the set number of objects are processed. For more information, see Windchill Search Properties.
* 
You can also use the Indexing Administration utility available from Site > Utilities > Business Administration. For more information, see Indexing Administration Utility.
8
Delete the bulk indexing list of objects.
* 
After executing this option, you need to manually delete the index data directory for that particular core in Search Index Server setup.
9
Verify index data.
Select this option to verify if the objects marked as “indexed” in Windchill are actually present in the indexed data. This option is particularly useful while restoring Windchill/index folders.
* 
Use this option periodically (ideally, every 3-6 months) to ensure the correct index status of Windchill objects.
* 
Option 9 marks objects not present on the index server as failed.
Use option 7 if you want to check the status of your indexed objects.
Use option 4 to index any failed objects.
10
Back to Core Selection Menu
Use this option to select the core from the list of available cores which were installed at the time of Index Search Server configuration with Windchill server.
11
Exit