MiniScrape

Simple webpages scrapper written in GO

Installation

Clone the repository

git clone https://github.com/pestanko/miniscrape.git

Enter the clonned repository

cd miniscrape

Install the dependencies

go get .

Build the scraper

make build

Run the scraper

go run main.go scrape

Scrape the single webpage:

# For food category
go run main.go scrape -C food -N ubaumanu

Run the server

make run-serve

Add/Edit available webpages

The webpages list is located in ./config/default.yml.

License

Miniscrape is released under the Apache 2.0 license. See LICENSE