Anth's Computer Cave Projects

AaimiSiteMod. Website-wide file-modification tool

16th September, 2016

AaimiSiteMod is a Python tool to perform server-wide modifications on web pages. This allows you to modify standard features on hundreds of web pages at a time to deal with unexpected changes to your website.

Ideally you'll have designed your website in a way that doesn't require changing every-single page to make general site-wide changes. You can generally achieve this using CSS for layout, and loading dynamic content with PHP or JavaScript. It is difficult, however, to cater for every possible scenario, and here's where AaimiSiteMod comes in.

Usage scenarios

Let's say you write a lot of articles, and for some topics you like to include helpful links to a specific external website to help your readers. Now imagine that website shuts down, and you have an unknown number of articles with links helpfully pointing to a site that doesn't exist. If you only have a few articles you could easily open each file and change the link to an alternative site. But what if you have hundreds of articles?

Another scenario would be a product re-branding. If you decide to rename a product you feature heavily on your website, I can take hours of work combing through each page, and you are almost certain to miss a few instances.

The first task I set AaimiSiteMod was to modify links on Anth's Computer Cave as part of the transition to HTTPS. The program crawled every page in the Cave and converted every old 'http://anthscomputercave.com' link to 'https://anthscomputercave.com'. Although there were only one or two of these old http hard links on each page, changing these manually on 120 pages would have taken me hours and invited typos and other mistakes. AaimiSiteMod ripped through the hundred-odd pages in a couple of minutes.

Sure, changing 'http' to 'https' on specific links is fairly basic, but you can find and substitute complex strings, even entire blocks of code in some cases, like nav panels and forms.

Configure AaimiSiteMod

Note that AaimiSiteMod, like any program that automatically modifies files, can cause major harm if used incorrectly. If you don't have a full backup of your website, you should create one before using this program.

This is BETA software and you should first try the program with a copy of your website files before using it on your actual website.

That's the scary stuff out of the way. If you're still game you can download AaimiSiteMod here.

Extract the aaimi_sitemod.py file to the web-root directory on your web-server.

First you need to tell AAIMI whether you have an existing list of the HTML files on your site. If not, AaimiSiteMod can create a list for you containing every page on your site. If you are using your own list it needs to be a plain-text file with the full path for each page, with one page on each line. It should look something-like the image below.

An AaimiSiteMod web page list. Picture: Anthony Hartup.
An example page-list to use with AaimiSiteMod.

Open aaimi_sitemod.py and go to line 28. Change the filemode variable to "new" if you want AaimiSiteMod to create a page-list, or "existing" if you already have a list.

Choosing a website page-list for AaimiSiteMod to load. Picture: Anthony Hartup.

If you are using an existing page-list change the filelist variable on line 29 to the name of your text file. Otherwise, choose a file name for the new list AaimiSiteMod will create.

The webroot variable on line 32 is the path to the web-root folder on your server.

Now you can add the strings you wish to change if found inside any of your web pages.

The target_string variable on line 43 is the exact string you wish to replace if found. The replacement string on line 45 is what will replace the target string.

AaimiSiteMod choices to modify strings within web pages. Picture: Anthony Hartup.

The condition variable on line 47 is a third optional string that, if enabled, must be in a line before AAIMI will replace strings.

Using my example of changing all HTTP anthscomputercave.com links to HTTPS, I needed to replace "http" with "https", but only for hard links to anthscomputercave.com. I didn't want the program changing every unrelated external link on the site, so I used the condition variable to make sure only the desired links were changed.

AaimiSiteMod example variables. Picture: Anthony Hartup.

I used "http" for the target_string, "https" for the replacement string, and the condition variable was "http://anthscomputercave.com". In this case, I could have also just used "http://anthscomputercave.com" as the target_string, "https://anthscomputercave.com" as the replacement_string, and left the condition string blank for exactly the same result. In other cases, the condition variable can really help.

For example, consider you have one phone number for all inquiries from customers, and there are various lines scattered around your web pages like "Call this phone number for support", and "call this same number for a quote", etc. Now imagine you wish to use a new, dedicated phone number for support queries, and the existing phone number just for quotation queries. In that case you would use your old number as the target_string, your new number as the replacement string, and set the condition as "support". This will leave all instances of the existing phone number except those that that have "support" on the same line. The support instances would change to the new number.

The final variable you may need to change is the run_mode on line 57. If this is set to "confirm", AaimiSiteMod will prompt for confirmation before applying changes to each page. You should definitely leave this in place until you have tested your AaimiSiteMod configuration. More on that in a moment.

AaimiSiteMod example variables. Picture: Anthony Hartup.

Running AaimiSiteMod

Now you have configured your variables and strings, lets take AaimiSiteMod for a spin.

In a terminal, navigate to the root directory of your website and type: python sitemod.py

If you have the run_mode variable set to "confirm", the first time AaimiSiteMod finds your target_string inside a page it will create a modified copy of the page. It will then prompt you for confirmation before it replaces the original. The prompt will contain the full path to the temporary file, so you can open and inspect the modified file to ensure the correct changes have been made before you proceed.

Running AaimiSiteMod in a terminal. Picture: Anthony Hartup.

If everything looks good you can enter "y" at the prompt to replace the original file and move on to the next. If you are certain your configuration is correct you can enter "a" to turn off confirmation and AaimiSiteMod will modify all remaining pages with no further input from you.

I you are not happy with the modification to the initial file, enter "n" to skip that file and move to the next, or "c" to cancel all operations. You can then revisit the string variables in the code before trying again.

Next for AaimiSiteMod

That's about it for using this version of AaimiSiteMod, but there are more features and capabilities on the way in the next version.

Expect to see selective bulk-file-renaming options, as well as support for other web-file types such as PHP and JavaScript. We'll also add options to replace multiple distinct strings at once, using multiple conditions and replacements.

Other features will include website-optimization functions, such as minifying code and automatically adjusting images. There will also be a link-checker to rid your website of broken links.

Cheers

Anth

_____________________________________________


Comments

Leave a comment on this article



About the Author

Anthony Hartup, creator of AAIMI, Estimcad and Anth's Computer Cave.
Anthony Hartup runs the AAIMI Project, a Python machine-interface platform.
He also codes for the Estimcad Project and ControlCadGUI.


Search the Cave with AaimiClip