2001-10: I took over as webmaster for NativeOrchid.Org and wrote the first of these tools. The most basic one is webput which does the uploading, by FTP, of all webpages that have changed in my local copy of the websites I look after; it has undergone many changes over the years, mostly because no two webhosting outfits ever do things in the same way, and the webhosting for NativeOrchid.Org has changed several times. Webput is now designed for the file-structure one sees at HostExcellence.Com (which will be very similar to what's seen at most any modern webhoster), and for redirects via .htaccess (it used to do them with META-REFRESH tags which only works for HTML files but works at a webhoster who doesn't let you to supply the Apache .htaccess file).
The weblinkcheck.cron script is another important labour-saver; it automates the checking of links, a task that is painfully tedious when done manually. It is something that too few webmasters ever do, whether with automated or manual methods, based on the broken links one encounters on the world-wide-web.
An entirely different tool is my sendtoElist script. There are many commercial products to solve the problem of managing an "elist" and sending periodic emails to the email-addresses on such a list, however here is a simple command-line tool to solve that problem.
ertools.zip -- zipfile containing all of the webmaster-tools, the general-purpose scripts, and the other scripts I've published on this site, for easy installation. See README.htm for more info on this collection of tools, and how to install them. And see Unix-tools for Windows to install for Windows.
Here are the individual shell (bash) scripts:
webput | -- FTP-upload of modified files to remotely-hosted website |
WEBPUT-LL | -- FTP-upload of modified files to remotely-hosted website -- the low-level part |
WEBVARS | -- needed by WEBPUT-LL |
webclean | -- needed by webput |
webmksubsets | -- needed by webput |
webnavreplicate | -- needed by webput |
sitemap-gen | -- needed by webput |
webcp | -- needed by webput (ER-specific) |
genByCaption-ER | -- needed by webput (ER-specific) |
genOrchidsBySpecies-ER | -- needed by webput (ER-specific) |
genFrontpageCandidates-ER | -- needed by webput (ER-specific) |
INSTALL-new-version-of-nopercart | -- needed by webput (ER-specific) |
mk-webalizer-ER-distribution | -- needed by webput (ER-specific) |
webmv | -- needed by rename with --web option (ER-specific) |
WEBLINKCHECK-LL | -- check for broken links in a webpage (general-purpose) |
weblinkcheck.cron | -- cron-task to find and repair broken and redirected links in websites; an example of how to use WEBLINKCHECK-LL |
webcommentcheck | -- check webpages for HTML-comments containing "--" (general-purpose) |
WEBGETLOGS.cron | -- cron-task to download access-log files |
FTP | -- needed by WEBGETLOGS.cron |
404report | -- needed by WEBGETLOGS.cron |
webalize-auto | -- invoked from WEBGETLOGS.cron |
webalize | -- needed by webalize-auto |
bk-auto.cron | -- cron-task to do daily backups |
bkall | -- needed by bk-auto.cron |
bk | -- needed by bkall |
bk-restore | -- restore file(s) backup up by bk |
sendtoElist | -- send supplied email to each member of supplied elist |
generateElistFromHISTORY+MBRS | -- needed by sendtoElist |
subsetMBRStoSMALL | -- needed by sendtoElist |
fixPeggyMBRS | -- convert XL to HTML in one-line-per-row form |
countMBRS | -- count members in membership-db |
ListSubtract-e | -- set-difference for an elist |
harvest-emailids | -- a "bot" to harvest email-addresses for targeted email-campaign |
uniquify-emailidlist | -- eliminate duplicates in an elist |
Most of these scripts will require some customization before being useful for someone else, and some are so highly dependent on features of my own website that it's hard to imagine them being useful to anyone else (these are labelled "ER-specific").
Some of these scripts use regular-expressions via grep and sed to select and modify HTML and will only function correctly on HTML written in a simplified subset of HTML. The comments will explain limitations of that sort -- please let me know if I've missed any.