Crawling

Technical SEO code focusing on Crawling.

Log File Analysis for SEO – Working with data visually

Link: https://canonicalized.com/log-file-analysis-seo/

By: Dorian Banutoiu on Canonicalized

Improve internal linking for SEO: Calculate Internal PageRank

Determine what pages on your site might be seen as authoritative by search engines by computing your Internal PageRank.

Link: https://searchengineland.com/improve-internal-linking-calculate-internal-pagerank-r-246883

By Paul Shapiro on https://searchengineland.com/

Crawling & metadata extraction with RCrawler

How to crawl a website using Rcrawler package

Link: https://www.gokam.co.uk/seo-crawling-metadata-extraction-with-r-rcrawler/

By: François JOLY

Classic Packages for Web Crawling

rvest

Perfect to crawl a few of URLs, easy to use css style or XPath selectors

Link: https://cran.r-project.org/web/packages/rvest/rvest.pdf

by Hadley Wickham

Rcrawler

Good to crawl a website and extract metadata

Link: https://cran.r-project.org/web/packages/Rcrawler/Rcrawler.pdf

By Salim Khalil