In the second segment of the post on Apache Nifi, we will build a simple flow. We will use the Apache Nifi instance that was installed in the earlier post. This simple flow will pick a file from a folder and drop the file to another folder. Translating that to Nifi parlance, the simple flow will have just three parts: a GetFile processor, a PutFile processor and a connector to connect the two processors. Following steps detailed here, let us start the Nifi and navigate to http://localhost:8080/nifi/. The Nifi GUI is shown below:
As a first step, let us create three folders called source, sink and inTransit under parent directory. So, the folder structure will like as shown below:
The reason we created these three folders is that we will be using these three folders for demonstrating the simple flow. Files from source folder will be picked up by Nifi and dropped into sink folder. Files in transit will be dropped into inTransit folder. On the Nifi GUI, drag a processor into the area below it as shown below:
Once we drag the processor, the Add Processor dialog box comes up as shown below:
In the Filter box, enter getfile as shown below:
This will bring up only one search result called GetFile. Select it and click on Add. The GetFile processor is added to the designer interface:
In the same manner, add PutFile processor. Now he interface will look as shown below:
Right click on the GetFile processor and select Configure as shown below:
On the Configure Processor window, set name to PickFile and click on Scheduling tab:
On the Scheduling tab, enter 10 sec to set the Run Schedule. Details about Run Schedule can be seen by hovering mouse pointer on the question mark next to the Run Schedule field name. Click on Properties tab:
On the Properties tab, set Input Directory to D:\\source\\. This the directory from which the file is picked up. Note that Keep Source File property is set to False:
Click on Apply to apply these changes on this processor. In the same manner, right click on PutFile and click on Configure Processor. On the Configure Processor window, set name to DropFile and check on check boxes next to failure and success under Automatically terminate relationships as shown below:
Click on Properties to set the Directory to D:\\sink\\. This is the directory where the file will be dropped. Click on Apply. Finally, hover the mouse on either of the processors on the yellow triangle to see the message that there needs to be a connection:
To solve this issue, hover the mouse on PickFile processor and click on arrow in the green circle as shown below and drag it onto the DropFile processor and release the mouse as shown below:
This will bring up the Create Connection window that has the details about the connection. Click on Add:
This will complete the connection between these two processors:
Note that the yellow triangle is replaced with a red square indicating that this processor is ready for Start operation.This completes the process of building a simple flow in Apache Nifi. We will take up running this simple flow in the next post
As a first step, let us create three folders called source, sink and inTransit under parent directory. So, the folder structure will like as shown below:
The reason we created these three folders is that we will be using these three folders for demonstrating the simple flow. Files from source folder will be picked up by Nifi and dropped into sink folder. Files in transit will be dropped into inTransit folder. On the Nifi GUI, drag a processor into the area below it as shown below:
Once we drag the processor, the Add Processor dialog box comes up as shown below:
In the Filter box, enter getfile as shown below:
This will bring up only one search result called GetFile. Select it and click on Add. The GetFile processor is added to the designer interface:
In the same manner, add PutFile processor. Now he interface will look as shown below:
Right click on the GetFile processor and select Configure as shown below:
On the Configure Processor window, set name to PickFile and click on Scheduling tab:
On the Scheduling tab, enter 10 sec to set the Run Schedule. Details about Run Schedule can be seen by hovering mouse pointer on the question mark next to the Run Schedule field name. Click on Properties tab:
On the Properties tab, set Input Directory to D:\\source\\. This the directory from which the file is picked up. Note that Keep Source File property is set to False:
Click on Apply to apply these changes on this processor. In the same manner, right click on PutFile and click on Configure Processor. On the Configure Processor window, set name to DropFile and check on check boxes next to failure and success under Automatically terminate relationships as shown below:
Click on Properties to set the Directory to D:\\sink\\. This is the directory where the file will be dropped. Click on Apply. Finally, hover the mouse on either of the processors on the yellow triangle to see the message that there needs to be a connection:
To solve this issue, hover the mouse on PickFile processor and click on arrow in the green circle as shown below and drag it onto the DropFile processor and release the mouse as shown below:
This will bring up the Create Connection window that has the details about the connection. Click on Add:
This will complete the connection between these two processors:
Note that the yellow triangle is replaced with a red square indicating that this processor is ready for Start operation.This completes the process of building a simple flow in Apache Nifi. We will take up running this simple flow in the next post
