Re: Mirroring PostgreSQL related content

[Darcy Buskermolen <darcy(at)wavefire(dot)com>]

On Friday 21 April 2006 12:21, Marc G. Fournier wrote:
> On Fri, 21 Apr 2006, Darcy Buskermolen wrote:
> > On Friday 21 April 2006 11:59, Marc G. Fournier wrote:
> >> On Fri, 21 Apr 2006, Joshua D. Drake wrote:
> >>> Jim C. Nasby wrote:
> >>>> After seeing reports of some very interesting sounding articles that
> >>>> were hosted elsewhere are now 404...
> >>>>
> >>>> Do we make any efforts to mirror the content of PostgreSQL related
> >>>> stories or articles? Presumably this would require contacting authors
> >>>> for permission, but perhaps we can find some enthusiastic newbies to
> >>>> do that...
> >>>
> >>> It is a heck of a lot more then that... most authors don't have rights
> >>> to allow postgresql.org to do that.
> >>
> >> Are there any good 'link checkers' that we could run on the site, say on
> >> a daily basis, that would check for failed links?  I normally use the
> >> web based ones, but for this we'd need something good to run from the
> >> command line ...
> >
> > wget, lynx, fetch.........
>
> I didn't know they had an 'auto check' mode taht could be run from cron
> ... can you let me know what args should be used to traverse the www site
> to report just those links that are dead?  then i can get this into cron
> ...

Sorry I misunderstood how you were trying to do it.  I thought we had a list 
of these URL's already that we just needed to stat in which case fetch -1 url 
2>/dev/null and checking for a non 0 exitstatus would sufice. If you are 
wanting to crawl the postgresql.org site or a portion of it looking for 
external links that are not valid, perhaps linkchecker 
(/usr/ports/www/linkchecker) will do what is intended.


>
> thanks ...
>
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy(at)hub(dot)org           Yahoo!: yscrappy              ICQ: 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
>                http://archives.postgresql.org

-- 
Darcy Buskermolen
Wavefire Technologies Corp.

http://www.wavefire.com
ph: 250.717.0200
fx: 250.763.1759