Category Archives: apache-nifi

Apache NiFi 1.6.0 Error Post Upgrade

Just upgraded to the latest and grates NiFi 1.6.0 and i am getting a strange error on a ExecuteSQL Processor.

error: cannot insert multiple commands into a prepared statement

So the ExecuteSQL uses a Database Connection Pooling Service that runs some queries against a database in this case Vertica(with the latest jar), and there are multiple sequential queries.

I managed to fix the issue by rolling back my Apache NiFi Install to NiFi 1.5.0 and is all normal now.

I can also replicate the error.

I actually made a short Video on the behavior of the ExecuteSQL on 1.5.0 vs 1.6.0.

Demo of Error

Also Link: https://youtu.be/FJPf7PDmTSE

Any ideas ? maybe a bug with new release ?

Thx

Oracle + NiFi => all fields converted to String

I'm using NiFi to transfer Oracle (11g) database tables to HDFS (Avro format).

Something goes wrong with the typing of the AVRO columns: they are all defined as String, even when the Oracle tablecolumn is of another type like Numeric, Timestamp of Datetime. Clearly, this is annoying :-)

After some googling, I found suggestions that the problem exists in the combination of certain Oracle JDBC drivers and the converter to Avro. However, I could not find a proper solution; does anyone know what JDBC (or other?) driver to use in order to have correct typing in the Avro output?

can not run Apache nifi on Windows

In my current project, I have been using apache nifi for some experiments purpose. When I have configured the first time. The nifi was running perfect (I was able to see the canvas, toolbar on Google Chrome when I typed http://localhost:8080/nifi/ ). But, after restarting my computer once, I am not able to run it.

When I type nifi.0.7.0\bin\run-nifi.bat command, I see the following message on the console.

2016-07-27 10:19:39,246 INFO [main] org.apache.nifi.bootstrap.Command Starting A
pache NiFi...
2016-07-27 10:19:39,246 INFO [main] org.apache.nifi.bootstrap.Command Working Di
rectory: C:\NIFI-0~1.0
2016-07-27 10:19:39,246 INFO [main] org.apache.nifi.bootstrap.Command Command: C
:\Program Files\Java\jdk1.8.0_102\bin\java.exe -classpath C:\NIFI-0~1.0\.\conf;C
:\NIFI-0~1.0\.\lib\jcl-over-slf4j-1.7.12.jar;C:\NIFI-0~1.0\.\lib\jul-to-slf4j-1.
7.12.jar;C:\NIFI-0~1.0\.\lib\log4j-over-slf4j-1.7.12.jar;C:\NIFI-0~1.0\.\lib\log
back-classic-1.1.3.jar;C:\NIFI-0~1.0\.\lib\logback-core-1.1.3.jar;C:\NIFI-0~1.0\
.\lib\nifi-api-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-documentation-0.7.0.jar;C:\NIF
I-0~1.0\.\lib\nifi-nar-utils-0.7.0.jar;C:\NIFI-0~1.0\.\lib\nifi-properties-0.7.0
.jar;C:\NIFI-0~1.0\.\lib\nifi-runtime-0.7.0.jar;C:\NIFI-0~1.0\.\lib\slf4j-api-1.
7.12.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Dsun
.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt
.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properti
es.file.path=C:\NIFI-0~1.0\.\conf\nifi.properties -Dnifi.bootstrap.listen.port=4
9823 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=C:\NIFI-0~1.0\bin\..\
\logs org.apache.nifi.NiFi

Could you please advise me what is the problem? and how could I resolve the issue?

Importing data from mysql to Hive/hdfs using Apache Nifi

am trying to import data from mysql to hdfs/hive using Nifi facing some challenges. please suggest me.

  1. QueryDatabaseTable ---mysql data
  2. ConvertAvroToJson --- output

    [{"emp_id": 467260, "emp_name": "Rob", "emp_age": 32}, {"emp_id": 467261, "emp_name": "Vijay", "emp_age": 32}, {"emp_id": 467258, "emp_name": "Jayaprakash", "emp_age": 26}, {"emp_id": 467259, "emp_name": "Kalyan", "emp_age": 32}, {"emp_id": 467262, "emp_name": "Andy", "emp_age": 20}, {"emp_id": 467263, "emp_name": "Ashley", "emp_age": 24}, {"emp_id": 467264, "emp_name": "Mounika", "emp_age": 24}]

  3. splitjson -- how to split json file into single flow files