[devel] sitemap on packages.altlinux.org
Igor Zubkov
igor.zubkov на gmail.com
Вт Июл 5 08:49:00 UTC 2011
Hi, All!
Вчера я переделал в prometheus2.0 ту часть которая долго меня не
устраивала. Это сайтмапы. Они теперь делаются по крону раз в неделю в
воскресенье. Выглядит это примерно так:
# date && bundle exec rake RAILS_ENV=production sitemap:refresh && date
Mon Jul 4 11:16:33 MSD 2011
+ sitemap1.xml.gz 50000 links / 8.93 MB / 236 KB gzipped
+ sitemap2.xml.gz 50000 links / 8.94 MB / 231 KB gzipped
+ sitemap3.xml.gz 50000 links / 8.96 MB / 236 KB gzipped
+ sitemap4.xml.gz 50000 links / 8.97 MB / 236 KB gzipped
+ sitemap5.xml.gz 50000 links / 8.91 MB / 241 KB gzipped
+ sitemap6.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap7.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap8.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap9.xml.gz 50000 links / 8.78 MB / 238 KB gzipped
+ sitemap10.xml.gz 50000 links / 8.77 MB / 237 KB gzipped
+ sitemap11.xml.gz 50000 links / 8.9 MB / 241 KB gzipped
+ sitemap12.xml.gz 50000 links / 8.92 MB / 239 KB gzipped
+ sitemap13.xml.gz 50000 links / 8.76 MB / 239 KB gzipped
+ sitemap14.xml.gz 50000 links / 8.85 MB / 236 KB gzipped
+ sitemap15.xml.gz 50000 links / 8.92 MB / 236 KB gzipped
+ sitemap16.xml.gz 50000 links / 8.95 MB / 230 KB gzipped
+ sitemap17.xml.gz 50000 links / 8.95 MB / 234 KB gzipped
+ sitemap18.xml.gz 50000 links / 8.97 MB / 238 KB gzipped
+ sitemap19.xml.gz 50000 links / 8.88 MB / 240 KB gzipped
+ sitemap20.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap21.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap22.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap23.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap24.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap25.xml.gz 50000 links / 8.92 MB / 241 KB gzipped
+ sitemap26.xml.gz 50000 links / 8.89 MB / 239 KB gzipped
+ sitemap27.xml.gz 50000 links / 8.76 MB / 240 KB gzipped
+ sitemap28.xml.gz 50000 links / 8.88 MB / 236 KB gzipped
+ sitemap29.xml.gz 50000 links / 8.91 MB / 234 KB gzipped
+ sitemap30.xml.gz 50000 links / 8.97 MB / 230 KB gzipped
+ sitemap31.xml.gz 50000 links / 9.01 MB / 236 KB gzipped
+ sitemap32.xml.gz 50000 links / 8.91 MB / 239 KB gzipped
+ sitemap33.xml.gz 50000 links / 8.86 MB / 240 KB gzipped
+ sitemap34.xml.gz 50000 links / 8.78 MB / 238 KB gzipped
+ sitemap35.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap36.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap37.xml.gz 50000 links / 8.77 MB / 237 KB gzipped
+ sitemap38.xml.gz 50000 links / 8.8 MB / 239 KB gzipped
+ sitemap39.xml.gz 50000 links / 8.92 MB / 241 KB gzipped
+ sitemap40.xml.gz 50000 links / 8.87 MB / 239 KB gzipped
+ sitemap41.xml.gz 50000 links / 8.76 MB / 239 KB gzipped
+ sitemap42.xml.gz 50000 links / 8.91 MB / 234 KB gzipped
+ sitemap43.xml.gz 50000 links / 8.89 MB / 233 KB gzipped
+ sitemap44.xml.gz 50000 links / 8.99 MB / 232 KB gzipped
+ sitemap45.xml.gz 50000 links / 9 MB / 236 KB gzipped
+ sitemap46.xml.gz 50000 links / 8.91 MB / 239 KB gzipped
+ sitemap47.xml.gz 50000 links / 8.83 MB / 238 KB gzipped
+ sitemap48.xml.gz 50000 links / 8.77 MB / 238 KB gzipped
+ sitemap49.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap50.xml.gz 50000 links / 8.78 MB / 239 KB gzipped
+ sitemap51.xml.gz 50000 links / 8.76 MB / 237 KB gzipped
+ sitemap52.xml.gz 50000 links / 8.83 MB / 240 KB gzipped
+ sitemap53.xml.gz 50000 links / 8.92 MB / 240 KB gzipped
+ sitemap54.xml.gz 50000 links / 8.84 MB / 238 KB gzipped
+ sitemap55.xml.gz 50000 links / 8.76 MB / 238 KB gzipped
+ sitemap56.xml.gz 14642 links / 2.62 MB / 66.8 KB gzipped
+ sitemap_index.xml.gz 56 sitemaps / 4.38 KB / 375 Bytes gzipped
Sitemap stats: 2,764,642 links / 56 sitemaps / 107m20s
Successful ping of Google
Successful ping of Ask
Successful ping of Bing
Successful ping of Sitemap Writer
To ping Yahoo you require a Yahoo AppID. Add it to your config/sitemap.rb with:
SitemapGenerator::Sitemap.yahoo_app_id = "my_app_id"
For more information see
http://developer.yahoo.com/search/siteexplorer/V1/updateNotification.html
Mon Jul 4 13:04:33 MSD 2011
#
Итого, это почти три миллиона страниц. Зато статика и отдаётся почти
мгновенно. И теперь положить сервер в аут вытягивая с него сайтмапы не
получится.
P.S.: приминаю заказы на то что ещё можно сделать на packages.altlinux.org.
--
Igor Zubkov
http://hi.im/ice
Подробная информация о списке рассылки Devel