Saturday 5 May 2018

Apache Nifi - V

In this post, we continue to explore transformation of formats of files in Apache Nifi. In the last post, we saw the transformation from CSV file format to JSON file format. In this post, we will just do the reverse. For all the work in this post, we will be using the latest version of Apache Nifi, 1.6.0

The steps are the same as in the previous flow with only a reversal of the properties of Record Reader and Record Writer properties in ConvertRecord Processor. The input for this flow will be the output of the flow in the earlier post. Note that even though the file name of output file from the earlier post remains the same as input filename, the format of the output file is JSON. The input file if opened in Excel will have content as shown below:














The flow is shown below:


























The warnings on the ConvertRecord processor is similar to the ones we saw in the last post. They are shown below:
























The properties of GetFile properties are shown below:



















The properties of ConvertRecord are shown below:




















The properties of JsonTreeReader are shown below:




















The properties of CSVRecordSetWriter are shown below:




















The properties of PutFile are shown below:



















Enable the JsonTreeReader, and CSVRecordSetWriterController Services are run the flow. The Data Provenance of input in ConvertRecord processor is shown below:
















The Data Provenance of output in ConvertRecord processor is shown below:









We can see that the format has changed from JSON to CSV format. This concludes this post