Colly Docs Articles Services Datasets GoDoc
Colly
Docs Articles Services Datasets
GoDoc GitHub

Introduction
How to install Getting started Configuration
Best Practices
Debugging Distributed scraping Storage backend Using multiple collectors Crawler configuration Extensions
Examples
Basic Error handling Login Max depth Multipart Parallel Proxy switcher Queue Random delay Rate limit Redis backend Request context Scraper server Url filter
Real life examples
Cryptocoins market capacity Coursera courses Factbase Google groups Hackernews comments Instagram images Openedx courses Reddit Shopify sitemap Xkcd store items

Storage backend

Colly has an in-memory storage backend to store cookies and visited URLs, but it can be overwritten by any custom storage backend which implements colly/storage.Storage.

Existing Storage Backends

In-Memory Backend

The default backend of Colly. Use collector.SetStorage() to override.

Redis backend

See redis example for details.

boltdb backend

SQLite3 backend

MongoDB backend

PostgreSQL backend

Contact Us Documentation Services GitHub

© 2025 Colly Team

Page Source
Site map