Earth Notes: On Website Technicals (2020-08)

Updated 2022-11-30 09:08 GMT.
By Damon Hart-Davis.
Tech updates: Review rework, CSS contain and large pages, AutoAds and floats, moar moves, reviews fixed, MODBUS et al, Brotli, FAQ droop.
tools
Placating Google Search Console by rewriting reviews, moving stuff from the old RPi to the new, and adding the first hint of Brotli support now that https is available...

2020-08-31: FAQ Off The Top Table

Screenshot 20200831 FAQ dropoff

There was a huge drop-off in FAQ rich results impressions for EOU between the 13th and 15th of July (~1000 to ~200 for those days). The peak seen on this snapshot is ~1400 impressions on 11th June, dropping to 15 on 29th August and down as low as 7 on the 21st!

It appears that Google is pruning the rich and FAQ / How-To results, maybe to reduce search results clutter.

2020-08-22: Brotli Precompression

I have now stopped using the RPi2 (which does not support Brotli) to serve any of the EOU site in any variant.

I am tentatively adding support for Brotli-precompressed files (for https). I have hand-compressed and checked in the Share42 JavaScript for desktop and lite for a nominal first-page-load 280 byte (~20%) saving over gzip:

2787 share42-3.js
1180 share42-3.jsgz
 900 share42-3.jsbr

I can see in the browser developer tools that the Brotli version is used over https.

A part of the configuration looks like this (preferring br over gz):

# Serve pre-compressed content (eg HTML) if possible.
# If client accepts brotli (br) compressed files...
RewriteCond %{HTTP:Accept-Encoding} br
# ... and if the pre-compressed file exists...
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME}br -s
# ... then send .XXXbr content instead of .XXX compressed on the fly.
RewriteRule ^/(.+)\.(html|css|js|xml)$ /$1.$2br [E=no-gzip:1,L]
# If client accepts gzip compressed files...
RewriteCond %{HTTP:Accept-Encoding} gzip
# ... and if the pre-compressed file exists...
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME}gz -s
# ... then send .XXXgz content instead of .XXX compressed on the fly.
RewriteRule ^/(.+)\.(html|css|js|xml)$ /$1.$2gz [E=no-gzip:1,L]

Though it seems to happen automagically, there is no harm in explicitly adding a Vary: Accept-Encoding header when doing the above rewrites, eg with config of the form:

<IfModule mod_headers.c>
  <FilesMatch ".(html|css|js|xml)$">
    Header append Vary: Accept-Encoding
  </FilesMatch>
</IfModule>

2020-08-21: MODBUS Moved

Today MODBUS stuff has been shifted over to the RPi3B, and there are some other odds and ends to do, and log files to stitch together carefully.

HTTPS for www.EOU

To extend the EOU certificate to allow https://www.earth.org.uk (and https://earth.org.uk):

sudo certbot -a webroot -i apache --cert-name amp.earth.org.uk -d amp.earth.org.uk -d m.earth.org.uk -d static.earth.org.uk -d www.earth.org.uk -d earth.org.uk

All versions of EOU (desktop/lite/AMP) are now available over http and https.

Now that the stand-alone static.earth.org.uk is no longer required, eg as a host for https image, audio and video targets, I am redirecting it to www again. (Such redirects for unfashionable aliases go to the https version, even though the canonical remains http for now.)

2020-08-20: Reviews Fixed

Screenshot 20200820 Review graph all fixed

All reviews are now converted to schema.org/Review nested within Product / Event / Book as appropriate to stop GSC complaining.

This has given rise to a curious inflation of somewhat spurious metadata in that what was one review has become (as reported in GSC) two Review 'fragments' and a Product/Event/Book, ie three items in GSC!

There is some extra metadata information in there, mainly brand and global identifier such as barcode where available. But I think that in trying to fix an index SPAMming issue, Google has created a new noise monster...

Anyway, no errors currently reported in GSC for EOU, hurrah!

In other news: a 5V supply has now been wired from the RPi3B to the dump load relay, so the RPi3B is controlling the load from ~19:30Z.

2020-08-29: after a bit of a delay, a page that I had forgotten popped out of the woodwork with an error in GSC, using hreview! And indeed it looks as if there's about another three pages to convert too...

Here is part of one of the hreview items to be retired:

<div class="hreview">
<ul>
   <li class="item"><span class="fn"><a href="https://www.google.com/drive/">Google Drive</a> - <q>Cloud Storage < File Backup for Photos, Docs < More</q></span></li>
   <li>Reviewed by: <span class="reviewer">Damon Hart-Davis</span> on
       <span class="dtreviewed">2017-08-05</span>.</li>
   <li class="summary">Generally reliable cloud backup and collaboration tool.</li>
   <li class="description">
       An application that integrates with my Mac desktop, and Firefox
       and Chrome browsers, making cloud backup easy and cheap with
       a bit of command-line stuff from me.  On top of that Google Docs
       is an excellent collaboration tool, and anything in one of Google's
       formats doesn't count towards the storage limit.
       ...
       </li>
   <li>Rating: <span class="rating">4.5</span> out of 5</li>
</ul>
</div>

2020-08-15: Moar Moves to RPI3B

I have now moved across the remaining hd.org sites, just as http (not https): www.hd.org, d.hd.org, aj.hd.org.

With Ansible and some templating, the process was a matter of minutes.

I also checked that the SunnyBeam works plugged into the RPi3B, ie that I could read a live power generation value from my grid-tied system. I waited until after dark so that I did not lose a reading on the live server.

On the 18th I integrated the new GPIO driver code into powermng and moved the control wiring across, but need to provide +5V to the driver/relay also.

2020-08-11: AutoAds and Floats Don't Play Nicely

20200811 screenshots AutoAds does not play nicely with floats

Google's AutoAds seems to have a problem where a float above it causes AutoAds not to push down the space created for an ad, but does push down the ad itself, obscuring following content.

I don't understand why such a basic thing should go wrong. I report the issue by closing/reporting the ad as "covering content". I don't know if that's the best way.

2020-08-16: I have started adding contain:content to the wrapper for the non-AMP explicit AdSense inserts, in preparation for turning AutoAds off again. This should be OK because I disable ads that expand outside their borders for example. The few pages I have rebuilt like this seem OK.

Also in preparation for turning off, or at least watering-down the effects of, AutoAds I have boosted the number of explicit ad slots on the most popular pages. There is less cause for AutoAds to insert anything at all.

2020-08-24: I'm seeing clipping of the sides of ads in narrow (portrait) mobile view, since the ads try to bust out of the container to the edges of the viewport.

So I have weakened containment to contain:layout to avoid this.

2020-08-07: CSS for Large Pages

I have added the facility to insert extra CSS support to improve browser performance of pages that are large (source over ~16kB for now) or that contain an insert, since and insert may be large and/or complex. I could modify that to be source+insert size over ~16kB for example.

This size/complexity threshold means that small pages which are fast anyway don't incur a size penalty for something that they don't need.

The first part of this simply attempts to make footers render a little more lazily on large pages where they won't be seen for a while, with footer { contain: content; }.

This constrains footers to be simple beasts with neither overflowing paint nor floats, basically, which is as they should be. They will not let things float into them on big pages, for example. But that will rarely actually make a difference nor be noticeable when there is a difference.

This initial implementation will only retain this CSS tweak if the body/insert text contains at least one footer, such as a Sources section, else the extra CSS will be stripped. The page wrapper text is not seen by the CSS minifier.

This large page support provides separate classes for layout and content containment. Parts of (for example) the home page can use content containment. All SECTIONs can use layout containment when not in desktop mode, but not content because full-width stuff busts out of the container's margins. The non-desktop part of this is implemented, and that is where it may make the most difference.

2020-08-08: the SECTION larding up with contain:layout now only happens (deep) below the fold. If in or near the viewport at the top of the page it would be unlikely to be able to help at all anyway. The containment is used in all page forms now, ie including desktop.

2020-08-05: CSS contain

I have experimentally applied contain:layout to the big table in the LED lighting page to try to speed up initial rendering. For that particular table contain:content (ie including paint) will not work because (on desktop) we bust out of its container's margins on purpose.

(Note: AMP objects to this styling on the div, so it's now arriving via the full-width table style.)

Further, because of similar margin-busting elsewhere, contain:layout is likely the most that can be applied blindly in most cases, and a clear:both before may be good to avoid astonishing float behaviour...

SECTION may be a good place to apply contain:layout routinely, maybe only below the fold, with some tweaks.

I am also testing contain:content on the home page for a couple of early simple blocks, and in a benign footer on all pages.

Argh: The property 'contain' in attribute 'style' in tag 'footer' is disallowed. (see https://amp.dev/documentation/guides-and-tutorials/develop/style_and_layout/style_pages/) Nor does AMP like contain on div. The issue seems to be the style attribute itself: CSS contain can be applied in other ways, eg footer { contain: content; }.

I have also applied contain:layout to all EOU's 'full width' CSS styles, both to potentially speed up layout, and to make such 'fullw' objects less likely to interfere with surrounding page elements. (AMP does not seem to be objecting to these.)

2020-08-04: Reviewed

hreview went out of fashion 20200609, switching to new Review nested in Product starting 20200803

Over the last few days I have made a big change in how (schema.org) Review snippets are implemented, because Google stopped allowing them as children of Article and similar, to reduce SPAMming of its SERPs. GSC (Google Search Console) switched to reporting all such Review snippets (and older hReview versions) as in error.

I now have such Reviews nested inside Product (or Event or Book), with the LED lighting page getting a major face-lift in passing.

~1638 words.