Earth Notes: On Website Technicals (2024-02)Updated 2024-02-26 20:37 GMT.
By Damon Hart-Davis.
This evening I did
brew install procmail on my Mac, and so have access to the real
lockfile. Which means that I can suddenly run my builds in parallel, eg
make -j 8 ..., taking full advantage of the M1's 8 cores!
2024-02-24: A11y: alt Preview Image Text
To improve Mastodon previews of tooted/posted articles I am adding
og:image:alt text for each
in article page headers.
This can make headers over-long, so is automatically dropped in that case.
I have extended the
alt text-from-image-name extractor to be able to use custom text from an
.alt.txt file where extant.
2024-02-25: all but ~6%
After a full site rebuild, fewer than 30 of ~417 desktop pages are missing
og:image:alt that could reasonably have it.
(A bit of editorial rejigging of titles and sub-heads and image names and alt text, and a few more pages have come into the fold...)
After the first auto-toot of an updated page, looking at the Web views of two Mastodon accounts, the server to which it was tooted (v4.2.7) does not show the alt text on hover, but a following server (v4.3.0-nightly.2024-02-23) does. Other toots on the first server seem to show the alt text however.
2024-02-23: Series RSS Feed
I have now made it easy to make an RSS feed for any page 'series', though for now I have set it up for Site Technicals.
The RSS feed file is created/updated as needed containing all the series pages, most recently updated first, and a link to the series head. Each series page gets built with a link to that RSS feed auto-inserted in the page header for auto-discovery with tools such as Brief, and in the series navigation for humans to discover/click manually.
I have also tentatively set this up for Saving Electricity.
2024-02-24: Desktop only
Just about as the complete site rebuild was finishing I decided that the auto-insertion of RSS links in series pages should probably be restricted to desktop full-fat pages only (not mobile/lite nor offline). Both to save some page weight but also for relevance: the feeds are of the desktop pages, less appropriate for other audiences.
2024-02-22: gCO2perkWh New! Improved! and Square!
By way of reducing Mastodon preview stampede load further from the regular grid intensity postings/toots, and to improve the preview look, I have created a smaller (by bytes and pixels) square
I have also added an
grid carbon intensity equation.
Except actually it's bigger on all counts! Try again!
This results in a (Mastodon v4.2.7) toot thus:
Old vs new:
7535 17 Sep 12:27 gCO2perkWh-1.png 4478 17 Sep 12:27 gCO2perkWh-1.png.webp 3545 29 Jan 18:08 gCO2perkWh-1.pngL 3484 30 Jan 13:06 gCO2perkWh-1.png.webpL 6883 22 Feb 10:47 gCO2perkWh-sq.png 3870 22 Feb 10:50 gCO2perkWh-sq.png.webp 1780 22 Feb 10:47 gCO2perkWh-sq.pngL 1496 22 Feb 10:50 gCO2perkWh-sq.png.webpL
I have annotated the home-page links to featured and popular articles with
as an experiment. (I have manually annotated one other link also.)
To allow this I additionally now have the home page declare itself to be of the
2024-02-15: Toot Lite When LOW
For once, an idea when falling asleep that turned out to be good... (Though I nearly forgot it entirely.)
When auto-tooting an EOU page to Mastodon, if the battery is
LOW then toot the 'lite' (
m.) page rather than the full-fat (
www.) desktop page. That will use a little less bandwidth etc at the point of tooting and the stampede to read the page, but also subsequently less for anyone actually visiting the page. Plus it shows off that EOU has 'lite' pages.
This will not happen outside of winter months, generally.
The test could be changed to
NOTHIGH, at which point some non-daytime toots would be lite too.
2024-02-12: Intensity Data Source Upgrade
Elexon will be turning off the unfashionable BMRS data service at
later this year.
In its place is a shiny new API at
of which the bit I am interested in (
FUELINST) has an
endpoint [with] optimised JSON payload and is aimed at frequent requests
datasets/FUELINST/stream) which sounds like the thing that I should use so as to be a good citizen.
After some faffing I have selected the public domain
JSON implementation for now, to parse inbound data.
I think that internally I will convert it to the old format for processing and cacheing initially. I can make some quick optimisations such as only requesting data since the last value already held if more recent than 24h.
By avoiding doing my research and manuscript correction work, I think that I have finished the upgrade to use the new JSON source.
I think that I will keep testing it in the IDE manually, and look for glitches, but it can probably go live in a day or two.
Lots of old commented-out code currently lurks: that should be cleaned up.
I was able to make an optimisation to request far less data from Elexon each time, typically well under an hour's worth even being quite cautious, and this can likely be trimmed further.
Initial updated version (not shrunk with Proguard yet) released at V1.4.0.
At the start of the long store the old data nominally arrived in this CSV form from Elexon. At the very end of the long store it is synthesised from JSON.
% gzip -d < _gridCarbonIntensityGB.longstore.csv.gz | head HDR FUELINST,20240209,23,20240209111000,8521,0,956,3793,12456,36,709,100,347,994,0,1055,0,3177,999,998,0,1397,1412 FUELINST,20240209,23,20240209111500,8514,0,958,3798,12320,0,709,99,394,995,0,1056,0,3177,999,998,0,1397,1412 FUELINST,20240209,23,20240209112000,8481,0,957,3792,12350,0,694,100,506,995,0,1055,0,3170,999,998,0,1397,1412 FUELINST,20240209,23,20240209112500,8466,0,946,3789,12415,0,692,100,528,995,0,1055,0,3164,1000,998,0,1397,1412 FUELINST,20240209,23,20240209113000,8446,0,944,3791,12485,0,692,100,540,995,0,1055,0,3166,999,998,0,1397,1412 FUELINST,20240209,24,20240209113500,8363,0,948,3794,12562,0,691,98,555,994,0,1056,0,3171,999,998,0,1397,1412 FUELINST,20240209,24,20240209114000,8285,0,945,3796,12603,0,692,99,562,994,0,1055,0,3172,999,998,0,1397,1412 FUELINST,20240209,24,20240209114500,8249,0,948,3800,12728,0,693,98,558,994,0,1056,0,3174,999,998,0,1397,1412 FUELINST,20240209,24,20240209115000,8129,0,951,3795,12929,0,694,98,530,994,0,1055,0,3179,999,998,0,1397,1412 % gzip -d < _gridCarbonIntensityGB.longstore.csv.gz | tail FUELINST,20240216,21,20240216102000,11645,0,229,3799,6099,231,735,0,233,2000,0,612,0,2260,999,997,0,1399,804 FUELINST,20240216,21,20240216102500,11396,0,233,3796,6204,212,730,6,254,2001,0,612,0,2259,999,997,0,1398,804 FUELINST,20240216,21,20240216103000,11267,0,228,3793,6349,161,729,43,227,2000,0,612,0,2262,999,997,0,1398,804 FUELINST,20240216,22,20240216103500,11096,0,249,3786,6484,60,731,44,305,2000,0,612,0,2266,999,997,0,1398,804 FUELINST,20240216,22,20240216104000,11218,0,254,3777,6456,0,730,44,347,2001,0,612,0,2272,999,997,0,1398,804 FUELINST,20240216,22,20240216104500,11306,0,240,3780,6470,87,731,44,287,2001,0,612,0,2279,999,997,0,1399,804 FUELINST,20240216,22,20240216105000,11368,0,232,3787,6430,102,729,45,249,2001,0,612,0,2256,999,997,0,1398,804 FUELINST,20240216,22,20240216105500,11347,0,239,3788,6406,98,725,44,280,2000,0,524,0,2234,999,997,0,1398,804 FUELINST,20240216,23,20240216110000,11315,0,238,3785,6342,95,708,44,321,2001,0,365,0,2257,999,997,0,1398,804 FTR,2015
I fixed the Proguard minified JAR build by fixing the JRE used to run
108038 16 Feb 10:44 .work/reutils-1.4.0.jar 66410 16 Feb 15:44 .work/edhMain.reutils-1.4.1.jar
I trimmed still further the data requested from the Elexon server each time. I am now typically requesting 4 intervals of generation data rather than 287, (~70x down) which feels like a win. Technically 2 of those are still redundant, but I am being cautious and allowing for clock skew and other funnies...
I commented out or removed more historic Twitter support code and libraries. (This de-Twitter-ing process started around 2022-11-11.)
2023-02-17: reliable so far
The have been zero drop-outs in gathered
FUELINST data in the approximately 24h the new system has been in place (to 17:00Z). There were three drop-outs on the morning of the 16th under the old system.
2024-02-07: Intensity Page Tweak
I made some tiny tweaks to the GB Grid Intensity page, removing some redundant stuff including Twitterisms, by editing the properties file and waiting for a recalculation/rebuild.
I validated the live HTML with the online Nu Html Checker, as usual, which pointed out some further characters to excise...
(Yes, this was "testing in production", so sue me!)