International Journal For Multidisciplinary Research
A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal
Plagiarism is checked by the leading plagiarism checker
Volume 5 Issue 5
Pattern Finding In Log Data Using Hive on Hadoop
|Abstract||Web log file, in the computing context, is the log file which get routinely generated and maintained by a web server. Analysing web server access logs will give information regarding user’s behavior. Log files generate data which contain valuable information from the user which get stored in the web server. Server logs act as a guest sign-in sheet. Log files give information about the pages which had a heavy traffic and least. What sites refer visitors to your site? What pages that your visitors view? Because of the tremendous usage of web, the web log files are growing at faster rate and the size is becoming huge. Processing this explosive growth of log files using relational database technology has been facing a bottle neck. To analyse such large datasets we need parallel processing system and reliable data storage mechanism, Big data uses the Hadoop where massive quantity of information is processed using cluster of commodity hardware. In this paper we present the Hadoop framework for storing and processing large log files and also analysing through hive, Hive is used in pre-processing of voluminous of log files and help us to find out the statics present in website and which help in our learning too.We can also perform optimization on hive query and we also compare the performance of both the analytical tools on analysing log files.|
|Keywords||Hadoop, data mining, log file analysis, behaviour mining, web mining|
|Field||Computer > Data / Information|
|Published In||Volume 1, Issue 2, September-October 2019|
|Cite This||Pattern Finding In Log Data Using Hive on Hadoop - Swapna Sahu - IJFMR Volume 1, Issue 2, September-October 2019.|
CrossRef DOI is assigned to each research paper published in our journal.
IJFMR DOI prefix is
All research papers published on this website are licensed under Creative Commons Attribution-ShareAlike 4.0 International License, and all rights belong to their respective authors/researchers.