Bulk Index Tool
When indexing is enabled and operating correctly, an indexable object is indexed as soon as it is created. However, there are times when you need to index large amounts of data at one time.
You can use the Bulk Index Tool to load Windchill Index Search libraries and their objects:
• Build indexes of existing data that belong in an index according to index policy.
• Start and stop the bulk indexing process.
• Schedule the process to start and stop at specified times.
• Check the status of the overall bulk indexing process.
• Attempt to re-index objects that have failed the indexing process.
• Maintain a detailed log of the indexing process.
• Reinitialize a Windchill Index Search library after changes have been made to the indexing policies.
• Reinitialize a Windchill Index Search library after changes have been made to the publishing policies.
Because loading indexes can take a significant amount of time, it might be necessary to stop the operation for some length of time. The index state is maintained in the IndexStatus table. This allows you to stop and restart the bulk indexing process without re-indexing objects.
The basic process of the Bulk Index Tool is as follows:
1. Query Windchill for all applicable objects.
2. Compare those objects to the IndexStatus table to determine if they have been indexed.
3. Determine whether each object belongs in a collection according to the index policy of the domain to which the object belongs.
4. If applicable, the object is indexed into the appropriate collection.
 |
Windchill only indexes the latest iteration of any revision.
If a user searches for the latest iteration of an object that was loaded using the data loading utilities, all iterations appear in the search results. You can correct this problem by using the Bulk Index Tool to re-index the data once it has been loaded.
|
Prerequisites and Performance Improvement Tips
• To improve performance, disable indexing when bulk loading data. Once the bulk loading operation is complete, re-enable the Bulk Index Tool and populate your indexes with the new data.
• If you plan to modify the default MIME file types for content indexing, follow the procedure outlined in Specify MIME Types for Content Indexing prior to running the Bulk Index Tool.
• If you are indexing large amounts of data comprised of content holder objects (such as documents), consider setting the wt.index.BulkIndexContentQueueList property. This property establishes separate indexing queues for content holder objects. For more information, see Windchill Search Properties.
|
|
If bulk indexing is already running on the server, then before setting this property you must delete and recreate the existing indexing queue entries. To do this, select option 2 from the Bulk Index Tool to stop bulk indexing. After some time has passed, use the Queue Management utility to verify that all indexing queue entries have been cleared.
|
Solr Commit Frequency
To reduce the frequency of Solr commits, increase the amount of time between auto-commit events:
1. Navigate to the following file:
<Index_Search_Home>\SolrServer\solr\solr-home\wblib\conf\solrconfig.xml
2. Increase the maxTime value from 60000 ms to 300000 ms:
<autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>300000</maxTime>
</autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>300000</maxTime>
</autoCommit>
3. Restart the server and start indexing using the Bulk Index Tool.
4. When bulk indexing is complete, revert the maxTime value back to 60000 milliseconds:
<autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>60000</maxTime>
</autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>60000</maxTime>
</autoCommit>
For more information, see autoCommit.
Using the Bulk Index Tool
To use the Bulk Index Tool, you must log in as an administrator.
Open a Windchill shell and enter the following command:
windchill wt.index.BulkIndexTool
If you are performing bulk indexing on a server that has both Windchill Index Search and Windchill PartsLink installed, the Bulk Index Tool prompts you to select a core library (collection). The default library is wblib.
 |
You can open two command prompts, side by side, to simplify the process of running the tool. Use one command prompt to run the Bulk Index Tool and the other command prompt to tail the BulkIndexTool.log file.
The tail utility is a standard UNIX utility. This utility is also available for Windows from GNU at the following website:
For real-time progress, you can run the tail utility on the BulkIndexTool.log file. For example:
tail -f BulkIndexTool.log
|
Bulk Index Tool Menu Options
|
0
|
View the current value of the following properties:
• wt.index.enabled
• com.ptc.windchill.partslink.index.enabled
• wt.index.bulkIndexSize
• wt.index.BulkIndexQueueList
• wt.index.BulkIndexContentQueueList
• wt.index.maxContentSize
• wt.index.checkDocumentTypes
• wt.index.filterFileTypes
• wt.index.IndexAndSearchMetaDataOnly
• wt.index.checkIndexingRulesBeforeQueue
• wt.index.excludeAttributes
|
||||
|
1
|
Start the bulk indexing process
Select this option to begin indexing your data. This option also creates an entry in the BulkIndexQueue, which executes the actual bulk indexing task.
If you have previously started the bulk indexing process and it is still running when you select this option, you receive an error message.
|
||||
|
2
|
Stop the bulk indexing process.
Select this option to stop the bulk index loading process and remove any remaining bulk indexing queue entries.
|
||||
|
3
|
Schedule the bulk indexing process.
Select this option to set up a regular schedule to repeat the bulk indexing process. You might want to schedule this time during low user activity.
Enter the following information:
• Start time. Use the format mm/dd/yyyy hh:mm am/pm.
• Stop time. Use the format mm/dd/yyyy hh:mm am/pm.
• Total number of runs (how many times you want the scheduled task repeated).
• Frequency (in days) that you want the bulk indexing task to run. (For example, enter 1 for daily; enter 7 for weekly.)
|
||||
|
4
|
Reset failed entries
Select this option to reset the objects that failed during indexing. You can use this option to process failed objects again.
|
||||
|
5
|
Reset entries that are processing.
Select this option if you have objects that have not yet been marked as complete. This can happen if communication between the indexing engine and Windchill occurred and Windchill did not update the object.
|
||||
|
6
|
Reset entries that have no indexing policies.
Select this option if you have changed indexing rules and objects that were previously not indexed should now be included.
|
||||
|
7
|
Check the bulk indexing progress:
Select this option to view indexing status.
The following status example indicates that out of 3609 objects, 3588 are indexed, 15 objects failed, and 6 objects have yet to be indexed.
Current status of Bulk Index:
Total Objects Handles: 3609 Objects processed: 3588 Objects processing: 0 Objects w/o indexing policies: 0 Objects remaining: 6 Objects failed: 15 When all objects have been processed, the bulk indexing process is complete.
|
||||
|
8
|
Delete the bulk indexing list of objects.
|
||||
|
9
|
Verify index data.
Select this option to verify if the objects marked as “indexed” in Windchill are actually present in the indexed data. This option is particularly useful while restoring Windchill/index folders.
|
||||
|
10
|
Back to Core Selection Menu
Use this option to select the core from the list of available cores which were installed at the time of Index Search Server configuration with Windchill server.
|
||||
|
11
|
Exit
|
