Auto-updated 2021-10-26 10:42 GMT.
Earth Notes: On Website Technicals By Damon Hart-Davis.
Read my nitty-gritty tech updates and daily learnings while keeping the Earth Notes site up and running; site stats also.
All about the interesting (and mundane) behind-the-scenes tech bits and bobs to grow and optimise this site, including purely technical measures such as speed, but also user experience beyond just adding new articles and updating existing ones.
These notes will be updated intermittently, usually when I should be doing something else!
Please try the useful tools, reasonably recently used and re-used by me, and encountered while writing these entries, listed under
The Sources/Links section of each of the pages lists interesting and/or useful resources encountered, even if not directly used for the site, generally that month.
Also please take a look at the simple automatically-updated
site stats below.
I welcome feedback on any of the issues that I have discussed. I share this stuff because it's interesting and because writing it here may save someone else some unnecessary head-scratching! No longer do we need to really
know anything technical, nor own a shelf full of reference books (so '90s!), we just need to be able to compose queries for our favourite search engines!
The newest pages are at the top of this list, and the newest items are at the top of each linked page. Enjoy!
Notes On Site Technicals: Index
2021-10 crazy page experience.
2021-09 crazy page experience, settling, Save-Data automatic lite pages.
2021-08 improved cwebp, liter, lingering, Ko-fi, AVIF, boldness, JXL from JPEG, 304s?, rebuild speedup, AMP 0, keen img inline, repo move.
2021-07 AMP be gone, going, mmm, HTTPS m-dot, WEBP footling, AMP gone, webp lo-fi, not much Save-Data, yak shaving, ate my hamster.
2021-06 precise CSS minify, connection down, new series, down with AMP.
2021-05 LIFX, JXL, previous-article teaser.
2021-04 Ds are good, more tweets vicar?, Ds are fiddled with, moar, less, 2%, 2 brews, phone, MBA too, JXL, max green?, all zeroes.
2021-03 DNS primary fun, fast site, build faster in the sun, new dump scheme, flaky router, bylines.
2021-02 image preview tweak for dark mode, DNS secondary fun.
2021-01 new year data capture, min.js, hosting, soft params, profile opt, hot pages, storage, unLooped, loooong fsck, uptime, dark tweaks, INTIFA2.
2020-12 vignette ads, year-end to-dos, Big Sur and FTDI, half traffic, time travel, 20/20.
2020-11 work storage, Let's Encrypt auto-renew, lazy wins, slow https switch, AMP https only, soft canonical, Apple touch, Apache stop, ad sub.
2020-10 smaller than recommended, https 150ms slower, https Dataset canonical, textract, ORCID, 1995.
2020-09 Brotli side, AMP https preferred, H2 oddity, anchor ads away, forever compression, canonical https www, 92222, GSC domain property.
2020-08 Review rework, CSS contain and large pages, AutoAds and floats, moar moves, reviews fixed, MODBUS et al, Brotli, FAQ droop.
2020-07 micro-optimisation fun, mobile first, sizes is important, denser displays, MD5 names, AMP cert, m-dot move.
2020-06 VIDEO/AUDIO style responsive tweaks, AutoAds on again, CSS trim, Ansible, desktop minify, throttle, pop star, HTTPS, HTTP/2, ADC, RPi speed.
2020-05 lower-fi audio and video for AMP, hi-fi for hi-res screens, podcast RSS episode images, lazier heroes.
2020-04 Blue Yeti, reduced media preload, download means download, 48kHz podcast, Zencastr, mono marker, GSC soft 404s, stats.
2020-03 performance tweaks, aggressive lazy, ad load, coronavirus, even lazier.
2020-02 GSC Review annoyance, CSS dark mode, video captions, lazy loading, srcset issues.
2020-01 AdSense AutoAds and GSC speed oddities, newsflash snapshot, frugal. #hashtagMagic.
2019-12 Dataset search and dateModified, not lazy yet, newsflash, ad shift, GSC page speed report implausible.
2019-11 new Fairphone 3, MIDI data feed, GSC PageSpeed Insights.
2019-10 GSC enhancements, automating data archiving, podcast rash, PodcastEpisode, auto-abstract, Audacity transcript.
2019-09 lack of instant podcast fame, .wav from awk, 5 per day, charge profile, explicitly not.
2019-08 maybe lazy, spatial coverage and Google Maps, goodbye JSON, long path wrapped, podcasts, links out and left float.
2019-07 improved video support, HTTPS, search impressions vs clicks, FFmpeg vs AVconv, line-height.
2019-06 Google search favicon, loading=lazy, dateCreated for a few, podcast and other audio support, Audacity, video support.
2019-05 displaying coverage, build too slow, ISO 8601 dates, GSC FAQ report, How-To, dated Comment, networking.
2019-04 moar liter, bumpy indexing, copyrightYear fix, Schedule, HH:MM and spatial page metadata, notworking, vox pop, tap target size.
2019-03 403, 2xGZip, FAQPage mix-in, m-dot/AMP, embedded BlogPosting, representativeOfPage, AMP ImageObject, MachMetrics, HTTPS, DefinedTermSet.
2019-02 micro-optimisation, isBasedOn, misuse of link rel prev/next, AMP half-indexed, Google-, soft 404, 1990 style, desktop tweak, 60% AMPed.
2019-01 Happy New Ear, cssgip, work storage, AMP srcset, LEDs, details, 400kpx image warning, bad AutoAd, indigestion, multi-hero, OGP revisited.
2018-12 feeds, IMG beyond AMP, Gallery CMS, test cases, random rebuild order, speakable structured data, lighter 404, moar AMPy, featured snippet.
2018-11 shorter autogen-image path and hero weight limit, images and link rel for AMP, IMG alt and SVG.
2018-10 preparing for the new RPi3 with 256GB of microSD card and BBR, app inventory, Bing crawl efficiency, info image and AMP.
2018-09 data file Atom sitemap in robots.txt, Google Dataset Search, poetry, DataDownload, CC0 licence, About, AMP.
2018-08 PWA revisited, auto lazy loading, jumpy AutoAds, more content pyramid, CRP and efficient canonicals, custom 404.
2018-07 warming up to HTTP/2 and Brotli and HTTPS, and the rest.
2018-06 creating a skim-friendly content pyramid, and post GDPR-calypse.
2018-05 CSS box-shadow performance for mobile, dns-prefetch fail, micro-optimisations, GDPR.
2018-04 reading time, liter, jpegtran to jpegultrascan, Primitive, SVG, Save-Data, Sitebulb.
2018-03 Auto Ad imbalance, incremental build, readability, tags, ad borders, TechArticle and Report, SoftwareSourceCode.
2018-02 Bing head, a saved byte, boxed cols and rounded corners, Google AdSense Auto Ads.
2018-01 PSNR lo-fi PNG autogeneration, page media, secondary image, client hints.
2017-12 allegedly too little markup, bad traffic, big hero, base download ms, service worker no rel, jump-to.
2017-11 Googlebot warp space, image re-optimisation, even liter, defer, inlining, video.
2017-10 rounded corners, mobile usability, HTTP/2 vs mobile, bad bot, UnCSS tweaks, latency, unit tests, visuals, Save-Data header, lite vs mobile.
2017-09 ImageMagick 20 years, Brew, autogen banners, old eyes, optimised ads, mobile traffic, brotli, doctype, JPEG fun, purifycss, UnCSS, OnDemand.
2017-08 Atom sitemaps (un)pending, Googlebot bandwidth, HTML improvements, regex big beast hunting, heroes, Cache-Control, restart drill, minifying.
2017-07 ad injection, meta, static precompression, zopfli, HTTP/head response overhead diet, Bing Webmaster Tools, FeedValidator, Share42, utf-8.
2017-06 CDN revoked, structured data, 10 years old, XML sitemap at long last and lastmod, HTML5 conformance, PageSpeed.
Stats updated: 2021-10-26T10:42Z
Fraction bot hits 0.568
Fraction GET 200s 0.799
Fraction GET 206s 0.002
Fraction GET 301s 0.032
Fraction GET 302s 0.072
Fraction GET 304s 0.030
Fraction GET 403s 0.000
Fraction GET 404s 0.025
Fraction GET 416s 0.000
Fraction GETs 0.962
Fraction HTTP hits 0.349
Fraction HTTPS hits 0.651
Fraction human main-page GET 200s 0.058
Fraction human m/(m+www) main-page GET 200s 0.166
Fraction site hits amp/(amp+www) 0.013
Fraction site hits m/(m+www) 0.056
Mean human main-page transfer bytes m 11770
Mean human main-page transfer bytes www 15215
Mean transfer bytes amp 849
Mean transfer bytes m 8185
Mean transfer bytes www 63915
Unique human IPs/day equivalent 187
Server Temperature Stats
Server sampled CPU temperature this month
Stat Temperature Date
First 47.8°C 2021-10-01T00:00Z
Min 42.9°C 2021-10-24T11:30Z
Max 66.6°C 2021-10-03T11:45Z
Last 44.0°C 2021-10-26T10:30Z
Build Energy Stats
System energy system status when desktop pages (re)built.
Snapshot at: 2021-10-26T10:42Z
Status when pages built
Count Status 246 VH
Accessibility According To Actual People With Disabilities: this site's two main sins would seem to be walls of text and black-on-white; let's not talk about my long-sentence habit.
Optimising a website for users with anxiety.
Share42.com social sharing buttons: lightweight and non-tracking.
Google's Webmaster Central Help Forum.
Feed (eg Atom) validation and W3C's tool.
Google's Structured Data Testing Tool, and Does your page support rich results? which seems a closer reflection to GSC's view.
Google's Rich Results Test: as of 2020-07-15 the Structured Data Testing Tool reports that it is being retired in favour of this.
Yandex' Structured data validator.
Structured Data Linter.
Cross browser testing tools: browserstack.com, browserling.com, browsershots.org. ( hat-tip)
Optimizing the Critical Rendering Path.
Yellow Lab Tools: Online test to help speeding up heavy web pages.
WebPageTest for site performance testing.
PageSpeed Insights from Google, which will provide compressed/minified versions of assets, as well as reporting what should be fixed/optimised.
See also for speed/performance testing: Test My Site With Google, Pingdom Website Speed Test, GTMetrix, MachMetrics which helped me quickly see where some fat (~30% of page weight) needed trimming, dotcom-monitor, Akamai Mobitest, Geek Flare, Page Weight.
Nibbler: a free tool for testing websites across a number of aspects.
Screaming Frog SEO Spider: "The SEO Spider is a desktop program you can install locally on PC, Mac or Linux which crawls websites’ links, images, CSS, script and apps to evaluate onsite SEO."
SEO Web Page Analyzer with interesting point about value of link anchor text (eg when read out by screen reader) to be able to decide to follow the link or not.
MobileMoxie Page-oscope mobile page test.
Alleged 8 major Google ranking signals in 2017 and Google's 9 major ranking signals and Google’s 200 Ranking Factors: The Complete List: things to get right to have visitors actually come and read the content!
Check My Links Chrome extension to validate links from the current page.
CSS minification online at cssminifier.com, and the excellent comand-line (node.js) clean-css.
HTML minification at htmlcompressor.com: gives me courage to know what I can safely tune myself! Also see the Google-recommended Kangax HTMLMinifier; this tool in its command-line version is now used in generation of this site.
UnCCS and purifycss static analysis tools to allow trimming of unused CCS, per-page.
Progressive JPEGs and green Martians: smart use of progressive JPEG scan scripts.
Image compression online with TinyPNG (JPEGs and PNGs) or off-line (including losslessly) with zopflipng (or OptiPNG) for PNGs, and jpegtran for JPEGs.
ClipChamp online video compression.
ImageMagick portable command-line image processing suite.
W3C Validator and HTML outliner.
CSS validator, including embedded in an HTML page.
TagCrowd, Wordclouds: create custom word clouds from, eg, your Web own page.
Normalise characters in response to a W3C validation warning "Text run is not in Unicode Normalization Form C" with FileFormat.info.
Pixabay for some handy images: thanks!
Website Dimensions, Best Practices. Blocking robots on your web page – the list of 1800 bad bots.
TinEye reverse image search.
Free Security Tests.
Zencastr: High Fidelity Podcasting: includes a free tier.
Performance Budget Calculator.