Updated Output code and Readme

README.md

@@ -1 +1,36 @@
-# webcrawler
+# webcrawler
+
+## Instructions to run 
+1. Clone the repository
+```
+git clone https://github.com/AmanTahiliani/webcrawler.git
+```
+2. cd into the directory
+```
+cd webcrawler
+```
+3. Install the requirements
+```
+pip install -r requirements.txt
+```
+4. Run the shell script
+```
+./amantahiliani.sh 
+```
+
+Note: If you get a permission denied error, run the following command
+```
+chmod +x amantahiliani.sh
+```
+If it still doesn't work, run the following command
+```
+python3 MultiThreadedCrawler.py {seed_url} --pages_to_parse=1000
+```
+
+
+## Output
+The output is generated in the form of the following files:
+1. pages_per_second.png - A graph showing the number of pages crawled per second
+2. craw_ratio_table.png-   A table showing the ratio of pages crawled to pages discovered
+3. crawlspeed.png- A graph showing the crawl speed in pages per minute
+4. Keywords_Output.json- A json file containing the keywords and the urls they were found in along with the frequency of the keyword in the url