WEBVTT NOTE Transcript 2024-03-21 from FLAC by https://elvery.net/prototypes/transcribe/ on my MacBook Air M1 on maximum quality setting initially. NOTE Body edited. NOTE Apparently Transcobble's mind was blown at 05:23. NOTE The transcript completed when run at minimum quality setting, and the tail was grafted on (and edited)... 00:00.000 --> 00:02.000 Statscast 00:02.000 --> 00:07.000 [Intro Music] 00:07.000 --> 00:17.000 Hi, I'm Damon Hart-Davis, and welcome to Earth Notes podcast on all things eco and green and efficient, at home. 00:17.000 --> 00:32.000 10th June 2020 for May's stats, turn-of-the-month update of home energy and related stats and playing with May's data for 16WW (ie EOU Towers), 00:32.000 --> 00:36.000 also other significant UK numbers over the month-ish. 00:36.000 --> 00:39.000 First, a mystery noise. 00:39.000 --> 00:46.000 [mystery noise] 00:46.000 --> 00:51.000 No, not a rainstorm nor sizzling bacon, more later. 00:51.000 --> 01:01.000 We've had clean air, blue skies and record sunshine, though more likely driven by climate change acting on the jet stream than lockdown reducing pollution it seems, 01:01.000 --> 01:06.000 and a lot more video calls for work and school and virtual pub. 01:06.000 --> 01:18.000 There have been changes, including signs of a rebound starting, in water use, the morning peak is later, internet use, reshaped and reduced GB electricity demand, 01:18.000 --> 01:30.000 changed GB electricity generation, especially renewables and coal, even at home, record May PV generation here by 15% and low imports to boot, 01:30.000 --> 01:39.000 and mobility, London having rebounded from 8% of usual at the start of April to about 20% at the start of June. 01:39.000 --> 01:46.000 Given that much retail is restarting the UK, it seems worth turning some EOU ad slots back on. 01:46.000 --> 01:48.000 16WW stats. 01:48.000 --> 02:00.000 Given all the warm weather, it's not astonishing that our mains water temperature was on the warm side for May, at 17 degrees C in the middle of the month and 19 degrees C at the end. 02:00.000 --> 02:06.000 More typical for mid-May is a degree either side of last year's 15 degrees C. 02:06.000 --> 02:22.000 In terms of 16WW energy, PV generation was 24kWh per day, well up on the previous high of 21kWh per day, and a mean and original PV GIST prediction of more like 18kWh per day. 02:22.000 --> 02:27.000 This micro-generation system is now over a decade old with no sign of flagging. 02:27.000 --> 02:41.000 Self-consumption was 86%, which means that we're largely avoiding using the grid as a battery and reducing the flows inbound, mainly just leaving outflows spilling PV energy to grid and reducing our neighbours carbon footprints. 02:41.000 --> 02:55.000 Daily gross leky consumption of 7kWh per day, ignoring PV and batteries, is higher than I'd like, and may drop a bit with our younger one back at school two days per week, and thus the Xbox and TV on less. 02:55.000 --> 03:08.000 If our planned heat battery were in place, then I anticipate that basically all of our 5kWh per day of gas consumption for hot water would have been diverted from our 18kWh per day of electricity exports. 03:08.000 --> 03:15.000 That would further reduce grid flows and avoid us burning anything at home other than the odd slice of toast. 03:15.000 --> 03:31.000 Also, on the raw side, after about three months lockdown, according to xkcd.com, each of our 100,000 head hairs grows at about half an inch per month, or an inch total between them every minute, or five feet every hour. 03:31.000 --> 03:37.000 Lies, damn lies and statistics, but I do resemble those remarks right now. 03:37.000 --> 03:50.000 Audification. Today's data is also 16WW related, from the logs of the EOU web server that is powered with off-grid solar PV at home in my kitchen cupboard. 03:50.000 --> 04:01.000 My company was one of the first internet service providers, ISPs, in the UK in the 90s, and some of the very first traffic I watched coming across our first live connection was malicious. 04:01.000 --> 04:07.000 I've learned to separate much bad versus legit traffic by eye over the years. 04:07.000 --> 04:12.000 I thought maybe it should be possible to make that difference audible. 04:12.000 --> 04:16.000 Basically, the observation is that some traffic types have observable patterns. 04:16.000 --> 04:22.000 Human traffic will likely show some sort of daily cycle, e.g. less traffic in the middle of the night. 04:22.000 --> 04:35.000 Malicious traffic can be very intense and bursty, not observing any etiquette of pacing traffic to be kind to the server, and since a lot of it is automated, i.e. code running amok by itself, has little daily pattern. 04:35.000 --> 04:48.000 Well-behaved bots or spiders, such as those run by search engines, Google, Bing, Yandex, etc., are also running all the time never sleeping, but are also measured and smooth and gentle in their requests. 04:48.000 --> 04:59.000 If you look at the show notes for this episode, you will see that I have taken three weeks log data from May this year, all under lockdown, and compared it with two weeks data from April 2018. 04:59.000 --> 05:10.000 I have stripped out information that might identify a human visitor and applied some fairly simple classification of traffic to try to pick out subsets that fit some of the above categories. 05:10.000 --> 05:23.000 The results have been graphed and converted to audio in various ways. Here is what all of one week's traffic from May sounds like with each day's traffic sped up to become one second of audio. 05:23.600 --> 05:52.600 Every request made to the server is a hit, such as download of a page, and another hit for each of the images downloaded to be shown within that page for example. 05:52.600 --> 06:15.600 The traffic is counted in buckets one second long, of which there are 604,800 in a week. Most, more than 90% off, buckets empty, i.e. with zero hits. A tiny handful may be a couple per week, have over 100 hits, usually from a particularly nasty attack attempt, or me running some stress testing tool. 06:15.600 --> 06:28.600 Because the sites that feed these logs are quite small, the vast majority of buckets have 8 hits or fewer. Traffic is bursty. That was the mystery noise from the start of the episode by the way. 06:28.600 --> 06:44.600 I then pick out my own traffic, including the odd stress test of my own system. This has a strong daily cadence, including regular automatic refresh of a status page I keep an eye on, which refreshes faster when the off-grid battery is more full. 06:44.600 --> 06:54.600 This has been reported as sounding like creaking. 06:54.600 --> 07:13.600 Basically, anything that isn't me that announces itself as a bot in the "User-Agent", I have treated as such. This traffic is mainly polite and smooth, a fairly constant sizzle. 07:13.600 --> 07:32.600 Bad actors often masquerade as good actors, and don't set the fictional evil bit in their data packets to make it easy to shut them out. But bad actions are often clear, such as probing for weaknesses in popular servers and languages such as WordPress and PHP, not used by EOU. 07:32.600 --> 08:01.600 This traffic happens all hours of the day and night, and features some loud pops from rude, unreasonably intense brute force requests. Bad traffic is picked out as anything that requests something complex that doesn't exist, or is forbidden that a normal human visitor's browser simply wouldn't try, or tries an access type, post the DOU doesn't use, all bots and my own local traffic are excluded. Here is bad traffic for the same May Week. 08:01.600 --> 08:30.600 I deduce human beings to be behind traffic other than my own, or bots. The request valid pages and media files, e.g. images and mp3. This does exhibit a daily cadence, like small waves washing on a beach, though with some possible masquerading bad actor pops if you listen carefully. 08:30.600 --> 08:52.760 And here's a week from April 2018, bot-sizzle. Bad pops. 08:52.760 --> 09:22.600 Human waves. Own creeks with a bit of stress testing pop apparently. 09:22.600 --> 09:29.960 Finally, here is three weeks of glorious May 2020 lockdown traffic. Enjoy in the knowledge that 09:29.960 --> 09:52.440 when you fry bacon you may in fact be hearing bots and bad actors in the sizzle and the pops. 09:52.760 --> 10:03.080 Do please read my extensive show notes for graphs, more sounds, and generally quite 10:03.080 --> 10:08.280 a lot more poking around in the data, not to mention my wish list of other things that seemed 10:08.280 --> 10:13.400 interesting. There are also links to sources and other podcasts that I've been listening to. 10:13.400 --> 10:21.240 Mystery number. Last month's Mystery number stood at 34034 as I came towards the end of 10:21.240 --> 10:31.240 assembling that stats cast. By early afternoon of June 10th it was 34684, so that's 650 new commits 10:31.240 --> 10:37.240 to the underlying repo from one month to the next, an audit trail and emotional roller coaster rolled 10:37.240 --> 10:44.280 into one. Shout out. I'm aiming to do a handful of five minute podcast interviews with listeners 10:44.280 --> 10:50.600 over the next week or so to see if their lives are greener or less so under lockdown, and what the 10:50.600 --> 10:55.880 interviewees think will happen to their greener next. Please contact me if you'd be willing to take part. 10:55.880 --> 11:02.040 More generally, please contact me if there's something that you'd like to discuss on the podcast, 11:02.040 --> 11:05.880 maybe including you for a mini interview. Thank you. 11:05.880 --> 11:13.160 [Music] 11:13.160 --> 11:21.240 There's more on my Earth Notes Web site at Earth.Org.UK.