While creating partition on Hive, the job killed frequently with status KILLED. Then i have gone through the log file, have seen actual root cause of the issue.
Error 1)
#ERROR
1) Status KILLED
Aggregation is not enabled. Try the nodemanager at localhost:54455
2) return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
3) [Fatal Error] Operator FS_2 (id=2): Number of dynamic partitions exceeded hive.exec.max.dynamic.partitions.pernode.
Solution
error shows it exceeds number of partition on single node I just increased the no.of partitions pernode to 200, by default it is 100
error shows it exceeds number of partition on single node I just increased the no.of partitions pernode to 200, by default it is 100
root@solai[bin]# hive
hive> set hive.exec.max.dynamic.partitions.pernode=200
the above solution will apply only for the current session, if you come out / exit the session hive.exec.max.dynamic.partitions.pernode property will assign 100.
If you want the changes to be made permanent, open hive-site.xml file under $HIVE_HOME/conf and change.
root@solai[bin]# sudo vim.tiny $HIVE_HOME/conf/hive-site.xml
