This paper introduces FoCUS (Forum Crawler Under Supervision), a supervised web-scale forum crawler designed to efficiently gather relevant forum content with minimal overhead. Forums, despite their varied layouts and software packages, share similar implicit navigation paths via specific URL types that connect entry pages to thread pages. FoCUS reduces the forum crawling problem to recognizing these URL types and learns accurate regular expression patterns of navigation paths from an automatically created training set using results from weak page type classifiers. These robust classifiers, trained with as few as five annotated forums, can be applied to a large set of unseen forums. Test results demonstrate FoCUS's effectiveness, achieving over 98% effectiveness and 97% coverage across forums using over 150 different software packages.
-
Jiang, Jingtian, Nenghai Yu, and Chin-Yew Lin. "Focus: learning to crawl web forums." Proceedings of the 21st International Conference on World Wide Web. 2012.
-
Members
- Live Games
- JoelR
- Brian
- Como
- IC Essentials
- COSMIN
- Adriano Faria
- Nathan Explosion
- DawPi
- Square Wheels
- Auto Evoke
- YalcinA
- master963
- VAHID
- Chris Anderson
- Myr
- bernhara
- opentype
- ReyDev
- send2yoni
- A Zayed
- terabyte
- Dilip
- ZLTRGO
- adik
- eivindsimensen
- envy
- onlyME
- V0RT3X
- GazzaGarratt
- Analog
- Voyage
- Paul Kaiser
- N700
- Paul
- TracyIsland
- Andy Y
- Omar Barbeytia carretero
- JoeyM
- Ryancoolround
- rainx
- YourSharona
- Kentraiyle Robinson
- MichaelR
- Edward Ellas
- IPS THEME
- aXenDev
- PrettyPixels
- Denis Dyack
- Labis
- DursunKaptan
- MissB
- TheLlamaman
- aLEX49566
- Codepixel
- alsl sndnxnx
- burnyourfeelings
- isvans
- Marius
- Matt
- Thomas Taschler
- Surpac
- JoshB
- Ioannis D
- abobader
- Richard Arch
- bdmusic 24
- Majster87
- TomCat
- Pmw
- Torgeir Rui
- Kammer et
- Nicolas PC
- XwReK
- Claudia999
- Kirill Gromov
- Synergy
- bing11
- Marcin Martyniak
- ArashDev
- ali hagi
- StevenM
- NewVicious
- lukash
- Andhrafriends Admin
- Daffy
- hyprem
- GuitarGathering
- Tripp
- Askancy
- MLK
- Jelly Belly
- eveneme eveneme
- Nomad
- Morphe
- lordi
- shahed
- John Horton
- PayMap
- Serval