Sample Custom Processor Implementation
The diagrams below present a few sample implementations of the custom processors in DataFlowML pipelines. Your specific implementation depends on the design of your solution.
Create Dataset Processor
In this sample implementation, the pipeline reads CSV data from the HDFS and sends it to the Create Dataset custom processor. The custom processor creates an analytics-ready dataset and outputs a job ID. A Print Output processor prints the job ID to a log file. The newly-created dataset is available for use in Analytics Builder.
Asynchronous Prediction and Scoring Results Processors
In this sample implementation, the first pipeline reads test CSV data from the HDFS and sends it to the Asynchronous Prediction custom processor. The scoring job outputs a job ID. In a second pipeline, the job ID from the first pipeline is sent to the Scoring Results custom processor. It retrieves the CSV results and writes them into the HDFS.
Synchronous Prediction Processor
In this sample implementation, the pipeline reads test CSV data and sends each data record to the Synchronous Prediction custom processor. The custom processor outputs results directly without saving them to a job. A Print Output processor prints the scoring results to a log file.
