Currently working with Apache NiFi and I need to put together a custom processor. There are a number of moving parts to getting an Ubuntu system set up for this, hence this aide memoire as otherwise I’ll never be able to do it again! I don’t know if this is the most efficient or even the most correct but it did at least work for me.
Pretty much all modern distributions have Java 8 now so it should be just a case of selecting the correct package for installation.
Set JAVA_HOME to point to the installation:
tar xzvf apache-maven-3.3.9-bin.tar.gz
Add the location of the bin folder to the path:
Check for correct installation by running:
Now it’s time to build NiFi following the instructions here. Clone the repo and build with Maven:
git clone http://git-wip-us.apache.org/repos/asf/nifi.git cd nifi mvn -T C2.0 clean install
This will take a while. Time to make some coffee…
This process creates tar.gz file in nifi-assembly/target. Extract this in to a convenient location – this will be the NiFi installation location.
Custom NiFi Processor
A boilerplate NiFi processor can be generated with Maven following the instructions here. Go to a convenient location for the project and enter:
mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=0.4.0-SNAPSHOT -DnifiVersion=0.4.0-SNAPSHOT
This works for the current version which builds as nifi-0.4.0-SNAPSHOT so change the line above to whatever is appropriate in the future. After a bit, the program will ask some question which can be answered like this:
- groupId – use org.apache.nifi
- artifactId – this is the name of the project. I’ll use nifi-richardstech-bundle.
- version – the version of the project. I’ll use the dafault of 1.0-SNAPSHOT.
- artifactBaseName – basically the middle part of the artifactId – richardstech in this case.
- package – just use default.
Then press enter when it asks for confirmation and the project should be generated. To build, enter:
cd nifi-richardstech-bundle mvn clean install
The output is a nar file in nifi-richardstech-nar/target. This should be copied into the lib directory of the nifi installation.
Now it is time to get NiFi running. Enter:
Then it should be possible to access NiFi using a web browser and URL:
Drag the processor icon onto the graph area. MyProcessor should now appear in the list of available processors.
I wanted to use NetBeans as an IDE to develop the processor code. I downloaded NetBeans from here. I used the All option.
NetBeans comes bundled with Maven and it may not be the correct version. So go to Tools -> Options and select the Java icon. There is a field for Maven Home. Set this to the directory where the Maven files were extracted.
To bring the project into NetBeans, just use Open Project and select the directory where the processor bundle was created (nifi-richardstech-bundle in this case). Then NetBeans should be able to compile the processor project.