Earth Notes: On Website Technicals

Auto-updated 2024-07-13.
By Damon Hart-Davis.
Read nitty-gritty tech updates and daily learnings that keep the Earth Notes site up and running; site stats also.
tools 800w JA
All about the interesting (and mundane) behind-the-scenes tech bits and bobs to grow and optimise this site, including purely technical measures such as speed, but also user experience beyond just adding new articles and updating existing ones. Also often a little linkfest of interesting articles I have read, on a somewhat wider tech curriculum...

NEW: RSS feed

These notes will be updated intermittently, often when I should be working on something else!

Please try the useful tools, reasonably recently used and re-used by me, and encountered while writing these entries, listed under Sources/Links.

The Sources/Links section of each of the pages lists interesting and/or useful resources encountered, even if not directly used for the site, generally that month.

Also please take a look at the simple automatically-updated site stats below.

I welcome feedback on any of the issues that I have discussed. I share this stuff because it's interesting and because writing it here may save someone else some unnecessary head-scratching! No longer do we need to really know anything technical, nor own a shelf full of reference books (so '90s!), we just need to be able to compose queries for our favourite search engines!

The newest pages are at the top of this list, and the newest items are at the top of each linked page. Enjoy!

Notes On Site Technicals: Index

#86: 2024-07
RPI3B again, 512GB found, MIDI albums, Referer no help.
#85: 2024-06
byte trimming, Googlebot still rogue, bot funnel, defence trimming, attacked, link saturation, slow winter, worst bots, DDoS.
#84: 2024-05
RED 406 and 429, mod_dumpio, time hash, mod_log_forensic, Sec-CH-UA-Mobile, rate limit, NTP, feed diet, time series, rogue Googlebot.
#83: 2024-04
ORCID, RSS work storage, podcast images, transcripts, Apache 2.4 ETag bug, 406 and more 429, less AMP, cacheing tweaks.
#82: 2024-03
METERCHANGE, micro-optimisation, podcast WebVTT transcript, CORS, Opus 16kbps, 429, time cues, mp3L, auto dark mode, lite-only text.
#81: 2024-02
intensity page tweak, new data source and JSON, toot lite, significantLink, square gCO2perkWh, series RSS, Mastodon preview a11y alt text.
#80: 2024-01
data curation, sitemap ping gone, heating polling again, offline too eager, RPi upgrade, more citation, stampede DDoS.
#79: 2023-12
ShellCheck, 503 work storage, Dataset citation glitch, 2024 already, video pages lost, ffmpeg, intensities, INTVKL.
#78: 2023-11
GSC Dataset pickiness, Zenodo DOIs, microSDVC card prep and f3, CrowView, public data archive.
#77: 2023-10
AdSense cookie GDPR, ill SD card, icons cache, stale data flag, BibDesk, biber, Overleaf, MacTeX, howpublished, winterising, deTwittering.
#76: 2023-09
crashy, bandwidth, og:image, archivedAt, copyrightNotice, contentReferenceTime, cookie consent, immutable, RPi5, reutils.
#75: 2023-08
INTVKL, static content URLs, code coverage, main pages archive, embargoes.
#74: 2023-07
statsHouse generative music, bird dropping, dark mode choice.
#73: 2023-06
heat, music.
#72: 2023-05
bang, 512GB SD, one OS to run them all.
#71: 2023-04
sonifying data to house, Apache and CC, awk function, TiMidity++, MIDI deflate, Ableton, Bitwig, LMMS, mild.
#70: 2023-03
reducing storage writes from logging, slow random.
#69: 2023-02
Apache scripts, FOSDEM energy and Java, GraalVM AOT.
#68: 2023-01
energy stats, ASCII .bib, et al., toot from Java, undead, technically, ProGuard, Sendmail HELO, bib lite, date +h, 410.
#67: 2022-12
citationID, Mastostorm, server down, mail system move, dataset archive, database cross-check, static Gallery, Xmas slump.
#66: 2022-11
tooting for climate, WiFi dongle swap, a11y tagging, BibTeX bibliography.
#65: 2022-10
SVG diagrams, spelling progress, title change, awl spelled rite, WebSite, Save-Data, GMT.
#64: 2022-09
TX down, spelingg, bad words, structured data.
#63: 2022-08
FUELINST glitch, review pros and cons, a11y, fanlife, Save-Data, traffic nadir, JXL.
#62: 2022-07
hot, cross (references).
#61: 2022-06
100% good page experience, conexDHW,, a11y, isAccessibleForFree, hunspell en_GB, throbber.
#60: 2022-05
timing tweets, energy stats insert, explosion, defer/async again, Airogram, off-grid, lowpowermode, stats fiddling, synthesis.
#59: 2022-04
grid support, GSC desktop crawl requests, HFS slow fsck, Air delay, new images.
#58: 2022-03
jq, daylight bug not saving.
#57: 2022-02
Eddi data and control, more not indexed, max immutable cache life.
#56: 2022-01
sitemap timestamp changes, power system tweaks, RE Utils V1.1.12, Christmas dip, desktop page experience, not indexed.
#55: 2021-12
Sitebulb 5.4.0 review, AMP off, reviewed reviewed, crawl frenzy, IndexNow.
#54: 2021-11
more meters, reutils tweets upgrade, DB-based Event and Product
#53: 2021-10
crazy inexplicable GSC page experience.
#52: 2021-09
crazy page experience, settling, Save-Data automatic lite pages.
#51: 2021-08
improved cwebp, liter, lingering, Ko-fi, AVIF, boldness, JXL from JPEG, 304s?, rebuild speedup, AMP 0, keen img inline, repo move.
#50: 2021-07
AMP be gone, going, HTTPS m-dot, WebP footling, AMP gone, WebP lo-fi, not much Save-Data, yak shaving, ate my hamster.
#49: 2021-06
precise CSS minify, connection down, new series, down with AMP.
#48: 2021-05
LIFX, JXL, previous-article teaser.
#47: 2021-04
Ds are good, more tweets vicar?, Ds are fiddled with, moar, less, 2%, 2 brews, phone, MBA too, JXL, max green?, all zeroes.
#46: 2021-03
DNS primary fun, fast site, build faster in the sun, new dump scheme, flaky router, bylines.
#45: 2021-02
image preview tweak for dark mode, DNS secondary fun.
#44: 2021-01
2022 data, min.js, hosting, soft params, profile opt, hot, storage, unLooped, looong fsck, uptime, dark tweaks, INTIFA2.
#43: 2020-12
vignette ads, year-end to-dos, Big Sur and FTDI, half traffic, time travel, 20/20.
#42: 2020-11
work storage, Let's Encrypt auto-renew, lazy wins, slow https switch, AMP https only, soft canonical, Apple touch, Apache stop, ad sub.
#41: 2020-10
smaller than recommended, https 150ms slower, https Dataset canonical, Textract, ORCID, 1995.
#40: 2020-09
Brotli side, AMP https preferred, H2 oddity, anchor ads away, forever compression, canonical https www, 92222[2], GSC domain property.
#39: 2020-08
Review rework, CSS contain and large pages, AutoAds and floats, moar moves, reviews fixed, MODBUS et al, Brotli, FAQ droop.
#38: 2020-07
micro-optimisation fun, mobile first, sizes is important, denser displays, MD5 names, AMP cert, m-dot move.
#37: 2020-06
VIDEO/AUDIO style responsive tweaks, AutoAds on again, CSS trim, Ansible, desktop minify, throttle, pop star, HTTPS, HTTP/2, ADC, RPi speed.
#36: 2020-05
lower-fi audio and video for AMP, hi-fi for hi-res screens, podcast RSS episode images, lazier heroes.
#35: 2020-04
Blue Yeti, reduced media preload, download means download, 48kHz podcast, Zencastr, mono marker, GSC soft 404s, stats.
#34: 2020-03
performance tweaks, aggressive lazy, ad load, coronavirus, even lazier.
#33: 2020-02
GSC Review annoyance, CSS dark mode, video captions, lazy loading, srcset issues.
#32: 2020-01
AdSense AutoAds and GSC speed oddities, newsflash snapshot, frugal. #hashtagMagic.
#31: 2019-12
Dataset search and dateModified, not lazy yet, newsflash, ad shift, GSC page speed report implausible.
#30: 2019-11
new Fairphone 3, MIDI data feed, GSC PageSpeed Insights, intensity log live, h3 tweak.
#29: 2019-10
GSC enhancements, automating data archiving, podcast rash, PodcastEpisode, auto-abstract, Audacity transcript.
#28: 2019-09
lack of instant podcast fame, .wav from awk, 5 per day, charge profile, explicitly not.
#27: 2019-08
maybe lazy, spatial coverage and Google Maps, goodbye JSON, long path wrapped, podcasts, links out and left float.
#26: 2019-07
improved video support, HTTPS, search impressions vs clicks, FFmpeg vs AVconv, line-height.
#25: 2019-06
Google search favicon, loading=lazy, dateCreated for a few, podcast and other audio support, Audacity, video support.
#24: 2019-05
displaying coverage, build too slow, ISO 8601 dates, GSC FAQ report, How-To, dated Comment, networking.
#23: 2019-04
moar litererer, bumpy indexing, copyrightYear fix, Schedule, HH:MM and spatial page metadata, notworking, vox pop, tap target size.
#22: 2019-03
403, 2xGZip, FAQPage mix-in, m-dot/AMP, embedded BlogPosting, representativeOfPage, AMP ImageObject, MachMetrics, HTTPS, DefinedTermSet.
#21: 2019-02
micro-optimisation, isBasedOn, misuse of link rel prev/next, AMP half-indexed, Google-, soft 404, 1990 style, desktop tweak, 60% AMPed.
#20: 2019-01
Happy New Ear, cssgip, work storage, AMP srcset, LEDs, details, 400kpx image warning, bad AutoAd, indigestion, multi-hero, OGP revisited.
#19: 2018-12
feeds, IMG beyond AMP, Gallery CMS, test cases, random rebuild order, speakable structured data, lighter 404, moar AMPy, featured snippet.
#18: 2018-11
shorter autogen-image path and hero weight limit, images and link rel for AMP, IMG alt and SVG.
#17: 2018-10
preparing for the new RPi3 with 256GB of microSD card and BBR, app inventory, Bing crawl efficiency, info image and AMP.
#16: 2018-09
data file Atom sitemap in robots.txt, Google Dataset Search, poetry, DataDownload, CC0 licence, About, AMP.
#15: 2018-08
PWA revisited, auto lazy loading, jumpy AutoAds, more content pyramid, CRP and efficient canonicals, custom 404.
#14: 2018-07
warming up to HTTP/2 and Brotli and HTTPS, and the rest.
#13: 2018-06
creating a skim-friendly content pyramid, and post GDPR-calypse.
#12: 2018-05
CSS box-shadow performance for mobile, dns-prefetch fail, micro-optimisations, GDPR.
#11: 2018-04
reading time, jpegtran to jpegultrascan, Primitive, SVG, Save-Data, Sitebulb.
#10: 2018-03
Auto Ad imbalance, incremental build, readability, tags, ad borders, TechArticle and Report, SoftwareSourceCode.
#9: 2018-02
Bing head, a saved byte, boxed cols and rounded corners, Google AdSense Auto Ads.
#8: 2018-01
PSNR lo-fi PNG autogeneration, page media, secondary image, client hints.
#7: 2017-12
allegedly too little markup, bad traffic, big hero, base download ms, service worker no rel, jump-to.
#6: 2017-11
Googlebot warp space, image re-optimisation, even liter, defer, inlining, video.
#5: 2017-10
rounded corners, mobile usability, HTTP/2 vs mobile, bad bot, UnCSS tweaks, latency, unit tests, visuals, Save-Data header, lite vs mobile.
#4: 2017-09
ImageMagick 20 years, Homebrew, autogen banners, old eyes, optimised ads, mobile traffic, brotli, doctype, JPEG, purifycss, UnCSS, OnDemand.
#3: 2017-08
Atom sitemaps (un)pending, Googlebot bandwidth, HTML improvements, regex big beast hunting, heroes, Cache-Control, restart drill, minifying.
#2: 2017-07
ad injection, meta, static precompression, zopfli, HTTP/head response overhead diet, Bing Webmaster Tools, FeedValidator, Share42, utf-8.
#1: 2017-06
CDN revoked, structured data, 10 years old, XML sitemap at long last and lastmod, HTML5 conformance, PageSpeed.

Site Stats

Stats updated: 2024-07-13T09:12Z

Fraction bot hits (target <0.5)0.590
Fraction GET 200s0.665
Fraction GET 206s0.001
Fraction GET 301s0.045
Fraction GET 302s0.073
Fraction GET 304s0.070
Fraction GET 400s0.000
Fraction GET 403s0.000
Fraction GET 404s0.017
Fraction GET 406s0.001
Fraction GET 416s0.000
Fraction GET 429s0.011
Fraction GET 503s0.031
Fraction GETs0.915
Fraction HEADs0.056
Fraction HTTP hits0.234
Fraction HTTPS hits0.766
Fraction human main-page GET 200s0.046
Fraction human m/(m+www) main-page GET 200s0.117
Fraction RSS podcast feed bytes RSS/www (target <0.01)0.004
Fraction RSS podcast feed hits RSS/www (target <0.045)0.085
Fraction site hits amp/(amp+www)0.031
Fraction site hits m/(m+www)0.058
Mean human main-page transfer bytes m20117
Mean human main-page transfer bytes www18967
Mean transfer bytes amp3003
Mean transfer bytes m18143
Mean transfer bytes www101450
Unique days seen13
Unique hours seen292
Unique human IPs/day equivalent153

Podcast Feed Pollers

Podcast RSS feed poll hits today/recent (13/Jul/2024) at 2024-07-13T09:12Z.

Estimated Hits per DayPartial User-Agent
45Amazon Music Podc
40PocketCasts/1.0 (
38Mozilla/5.0 (Wind
24Overcast/1.0 Podc
21iVoox Global Podc

Server Temperature Stats

Server sampled CPU temperature this month

Build Energy Stats

System energy system status when desktop pages (re)built.

Snapshot at: 2024-07-13T09:12Z

Status when pages built

Pages With Live Images

~2562 words.