About this page

Frontera: open source, large scale web crawling framework | PPT

https://slideshare.net/slideshow/frontera-open-source-large-scale-web-crawling-framework/53981994

“This document describes Frontera, an open source framework for large scale web crawling. It discusses the architecture and components of Frontera, which includes Scrapy for network operations, Apache Kafka as a data bus, and Apache HBase for storage. It also outlines some challenges faced during the development of...” (the page’s own description)

Topic: Not categorised yet
Quality: Not rated yet
Language: English
Text on page: 13,000 characters
Page size: 693 kB
Answered: OK (200), HTML
Last read: 6 Jul 2025
In our index since: 6 Jul 2025
Safe search: Not checked yet
Links to it: No other site we've read links here yet

Tools for this page

Other sites that can tell you more. Plain links: we don't track where you go.

About the site

Domain owner ICANN registration lookup
Certificates certificate transparency, subdomains
Safety check Safe Browsing status
Trackers The Markup's Blacklight
Is it down? for everyone, or just you

Discussions

Hacker News stories linking here
Reddit posts from this site
Wikipedia articles mentioning the site

For web makers

PageSpeed Lighthouse report
HTML validator W3C
Accessibility WAVE by WebAIM
Security headers

Who links to it

We haven't seen any links to this page yet. That usually means we haven't crawled the pages that link to it, not that there are none.

“Not rated yet” and similar notes are shown on purpose: we say what we haven't measured, so the page doesn't look emptier or better than it is.