Internet of Numbers

Size Frequency
1592161
10226968
10063194
100018037
10 0004942
100 0003617
1 000 0007089
10 000 0002804
100 000 0002227
1 000 000 0001044
10 000 000 000222
100 000 000 000145
1 000 000 000 0001386
10 000 000 000 000329
100 000 000 000 0001024
1 000 000 000 000 000+1758

This page is created from my web scraper that searches for numbers on the internet and records their frequency of the size. For example if it found a site with the numbers: 2, 5, 100. Then the 1 size frequency would increase by 2 and the 100 would increase by 1.

The web scraper works by loading a page. Scanning the body for links and numbers. It stores the numbers and waits 60 seconds before loading another page that was found on a previous page. After 30 rotations it updates the MySQL server by adding onto the values in the database. It then creates a text file from the database and uploads it to my server via a FTP protocol. The text file is then used by this php page to create the values. After 100 rotations it scraps all the upcoming URLS stored and selects a new random starting point.

Next time I would like to record the number of sites visited, add some logging methods and data usage.

The repository for the code can be found here - The Repository.

The author is Tom Bowyer. My website can be found here - His website.