Data Scraping

Building your own database from internet data goldmines

Data is gold to anyone doing business online and offline.

Google is a huge source of data having indexed 99% of an internet that gets bigger by the second.

For example, let's say you would like a spreadsheet with all webpages on Time Magazine's website. You can do a Google advanced search "site:time.com" and with a web scraper pull all information into your own database and mine it as you wish.

But what if you had a list of 10 or 100 or 1000, or 10,000 websites that you want in your database? With a little customization your web scraper can loop through that list of websites and pull every indexed webpage from all those sites and write it to your database.

Those are two examples of big data scraping that happens every day online. There's thousands of people that do nothing but scrape data, collecting various bits and pieces, assembling it into a massive unique and valuable databases, then reselling it.

I know because I ran a popular web scraper for internet marketers for 10 years. The crazy thing is I coded it while on vacation on a laptop as the same day the idea hit me.

Since then I have written countless little programs to pull information from hundreds of websites and data sources.

These programs are fast to develop, small and very focused applications that I've been building for 13 years.

It's a big data world and I've found that the types of databases that can be built are only limited by our imagination.

If you have an idea for something that would help you in your business or workplace, I'd love to hear about it. Email me at CoreyBornmann@gmail.com.

~ Corey Bornmann

The Case For A Better Content System

Delivering stories and making income doing it

7 minute read in Web Development

Read...


Mobile First Web Development

Building the fastest loading websites in the world

1 minute read in Web Development

Read...


Amigos


Luffa Sponges Don't Come From The Sea

Luffa's are gourds and we're growing them like crazy

3 minute read in Gardening

coreybornmann.com/stories/luffa-sponge


Klippyo

Create square videos, vertical videos, captioned videos, meme videos and compilation videos all from a single app.

2 minute read in Video

jvzalerts.com/stories/klippyo


Urban Garden First Fruit

A successful proof-of-concept mini urban garden

2 minute read in Real Estate

coreybornmann.com/stories/urban-garden-first-fruits