[2014-11-27 20:11:39] =-= User mode for DamonHD is now +iwx [2014-11-27 20:11:39] *NickServ* Welcome to ZJE, DamonHD! Here on ZJE, we provide services to enable the registration of nicknames and channels! For details, type /msg NickServ help and /msg ChanServ help. [2014-11-27 20:11:44] -->| YOU (DamonHD) have joined #opentrv [2014-11-27 20:11:44] =-= Topic for #opentrv is ``http://opentrv.org.uk This Channel is logged. '' [2014-11-27 20:11:44] =-= Topic for #opentrv was set by Gadget-Mac on 13 November 2014 19:21:26 [2014-11-27 20:25:17] evening DamonHD [2014-11-27 20:25:23] Hi there! [2014-11-27 20:25:38] right. give me 5 minutes to finish eating and grab a brew [2014-11-27 20:26:25] No hurry. [2014-11-27 20:26:31] fhuizing is here - he's interested in my comet based solution and he's working on an android front-end app that's pretty much what I had in mind [2014-11-27 20:27:05] Hello there [2014-11-27 20:30:22] DamonHD: anyone else coming? [2014-11-27 20:30:58] Dunno. [2014-11-27 20:31:16] I think Bruno is at someone's leaving do, and I can't summon Yodit... [2014-11-27 20:31:47] Well why not you start with fhuizing? [2014-11-27 20:32:17] ok. well we could have some preliminary discussions anyway. I've been busy evaluating a wifi module for work, but with a view to using it in the boiler controller [2014-11-27 20:32:37] it's a low-level one that will allow for dual wired/wireless without messing about with two totally different stacks [2014-11-27 20:33:02] This will be a continuing discussion, so we don't have to solve everything now! [2014-11-27 20:33:52] well I want to start specifying the tinyhan app layer stuff, in particular how to automatically map to mqtt topics, and also how to map node IDs to friendly names [2014-11-27 20:34:40] one thing in particular that needs some thought is whether data is grouped together or one data item per topic. I would favour the former I think because it would be significantly more efficient [2014-11-27 20:35:22] please speak up if you have any thoughts fhuizing because I know you have been thinking about that end of the stack [2014-11-27 20:35:53] Hmm, I don't know if this is the same at all but [2014-11-27 20:36:18] I am currently working on rotating the data fields as they change, and otherwise periodically [2014-11-27 20:36:34] to avoid transmitting everything every time for various reasons. [2014-11-27 20:37:16] what are the various reasons? [2014-11-27 20:37:32] there's nothing wrong with doing that, but I'm not sure it will be gaining much if power saving is your idea [2014-11-27 20:37:53] Well, conserving bandwidth for very slowly changing stuff live battery voltage [2014-11-27 20:38:18] Making it possible to send more parameters than can be sent in a small frame, esp in JSON. [2014-11-27 20:38:37] it's just another problem created by that json format [2014-11-27 20:38:43] And keeping individual frames short. [2014-11-27 20:38:51] by the time you factor in preamble, sync and header overheads adding a couple of bytes is nothing [2014-11-27 20:39:11] Yes, so in order to test my android client, I've created an emulator where one subscribes to per heater. "zhc/heater/+" which returns {curtemp, desiredtemp, valvepos, uuid} json encoded. [2014-11-27 20:39:35] B^> [2014-11-27 20:39:47] that sounds similar to what I was thinking fhuizing [2014-11-27 20:39:58] Although it might be desirable to subscribe per individual field. [2014-11-27 20:40:15] https://github.com/Error323/zhc/blob/master/emulator/zhc.py [2014-11-27 20:40:34] again I think if it's all related status data it's probably more efficient overall for the publisher to send everything at once, then the subscriber just to discard anything its not interested in [2014-11-27 20:41:14] there's a good argument for separating status, "permanent" config data and transient settings into a few separate topics I think [2014-11-27 20:42:02] Another more subtle issue that I am dealing with [2014-11-27 20:42:24] is that the number of fields you may want to send depends on the privacy level of the leaf, at least for now, [2014-11-27 20:42:39] so it is easier to allow a mechanism which allows a graceful 'overflow' IMHO [2014-11-27 20:43:03] mikestir, Indeed, I've also created a topic "zhc/static" [2014-11-27 20:43:19] tbh there doesn't have to be a direct relationship between what goes over the air and what ends up going to the broker [2014-11-27 20:43:50] Also very true [2014-11-27 20:44:03] Indeed it would be nice to keep things simpler downstream [2014-11-27 20:44:12] so if we thought some more about the concept of a tinyhan profile, where the "heating" profile describes a set of data that would be sent as json to the broker, the node could choose to send just a subset of values and the bridge would leave the rest unmodified [2014-11-27 20:44:15] something like that anyway [2014-11-27 20:44:19] by possibly 'demultiplexing' at the concentrator [2014-11-27 20:44:36] Yes. [2014-11-27 20:44:50] Now this is also neat for bandwidth optimisation even for binary. [2014-11-27 20:45:13] It feels a lot easier to meet duty-cycle limits this way to me. [2014-11-27 20:45:19] yeah, well the binary name/value approach allows for that easily [2014-11-27 20:46:01] So, we might agree that that demux / partial send approach would be a good basic idea. [2014-11-27 20:46:18] current sensor test for tinyhan takes less than 10ms to transmit a status update, so even 1 second updates is still ok for the 1% duty cycle sub-bands [2014-11-27 20:46:19] It does get more complex when the full set of fields is not known in advance. [2014-11-27 20:46:31] 0.1% for the one we're using. [2014-11-27 20:46:48] And I may wish to try to cram hundreds of devices into that band. [2014-11-27 20:46:58] 10 seconds is still 12 times more frequent than the conrad protocol manages :) [2014-11-27 20:47:08] Even with CS/CDMA I'd be happier just not sending stuff that hasn;t changed. [2014-11-27 20:47:09] why not use the 1% bit? [2014-11-27 20:47:13] is it necessary? [2014-11-27 20:47:27] 10x lower power limit? [2014-11-27 20:47:32] f: is what necessary? [2014-11-27 20:47:42] 10s interval [2014-11-27 20:47:58] DamonHD: it takes 160 us to send a byte. the vast majority of the packet is syncing up [2014-11-27 20:48:30] fhuizing: the interval can be a lot longer but the valves do need to call in periodically in case the coordinator has data waiting for them [2014-11-27 20:48:31] I hear you. My instincts are with what you are saying, by I still prefer to send very slowly changing values every time. [2014-11-27 20:48:52] but [2014-11-27 20:48:57] not [2014-11-27 20:49:01] BLAH [2014-11-27 20:49:15] My instincts are with what you are saying, but I still prefer NOT to send very slowly changing values every time. [2014-11-27 20:50:03] Shall I record the partial send idea as a decision for now, to be supported for binary and text formats as far as reasonably possible? [2014-11-27 20:51:31] yes, there's absolutely no reason why it can't be used in that way [2014-11-27 20:53:08] so thinking about the network organisation, I think we need some kind of friendly name so that the coordinator can map to a topic like rooms/lounge/heating rather than rooms/615488abb123411/heating [2014-11-27 20:53:19] Go on... [2014-11-27 20:53:32] (Update to page: http://www.earth.org.uk/OpenTRV-protocol-discussions-201411-2.html ) [2014-11-27 20:53:34] what I was considering was a special profile that would be used for node type identification and common data [2014-11-27 20:54:01] for nodes with a UI, like the comets, you could actually enter a short friendly name through the UI [2014-11-27 20:54:38] What I'd really like to get out of this whole schmole is pairing [2014-11-27 20:54:40] the system would allow you to push a longer one through from the network, or for nodes with no UI you'd have to do that after reading the unique ID off the device [2014-11-27 20:54:49] but once the friendly name is programmed it will always show up under that name [2014-11-27 20:54:59] so nothing like that *needs* to be entered on a tiny device... [2014-11-27 20:55:19] no, you could always fall back to looking for the hex string first time around [2014-11-27 20:55:27] So have a relatively-friendly autogenerated name or whatever [2014-11-27 20:55:39] yes I was just going to suggest that [2014-11-27 20:55:41] and then as you suggest push a nicer alias from the concentrator/Web end. [2014-11-27 20:55:44] just seed it with some dictionary word [2014-11-27 20:55:52] But I dunno. Glad you're thinking about it... [2014-11-27 20:56:17] so your network comes up rooms/sausages/heating and you do something like publish to rooms/sausages/config {"name": "lounge"} and you're away [2014-11-27 20:56:24] Hmm, I am not compressing the OED to fit into Flash! [2014-11-27 20:56:36] it would be a production operation [2014-11-27 20:56:48] eeprom programmed when firmware is loaded [2014-11-27 20:57:14] People have probably gotten used to entering paring codes or short alphanumeric strings [2014-11-27 20:57:34] the pairing thing is already kind of dealt with in the spec, although it would be easily spoofed until we sort out the authentication aspects [2014-11-27 20:57:52] so maybe it could be more like rooms/OpenTRV1-eA9343/heating for OpenTRV valves? [2014-11-27 20:58:10] the idea is to remember the uuid of the coordinator after first pairing and then only to ever attempt re-registration with that specific node unless some user reset operation is carried out [2014-11-27 20:58:20] From a user's perspective I don't see a huge benefit. It would be easier to perform that operation in the client? [2014-11-27 20:58:25] this is necessary to enable the network to rebuild itself under some circumstances [2014-11-27 20:59:08] fhuizing: which operation specifically? [2014-11-27 20:59:25] mapping of heaters to rooms [2014-11-27 20:59:35] or maybe I'm missing something [2014-11-27 21:00:10] it would be done in the client, but it's more how you identify the valves in the first place [2014-11-27 21:00:41] if you deploy 10 brand new valves and set the coordinator to allow attach then all you end up with is a load of hex digits [2014-11-27 21:00:55] you need some way of doing that initial name assignment [2014-11-27 21:01:45] whereas the comets have a 4 character alphanumeric interface, so you when you fit the valve you could enter "LOUN" "KTCH" "BED1" etc [2014-11-27 21:02:06] and then go and type in the proper name on the client, but without having to cross reference a load of hex digits [2014-11-27 21:02:25] Hmm, not going to be possible with minimialist OpenTRV UI! [2014-11-27 21:02:25] you could use the first 2 for the room, and the second 2 for the heater id [2014-11-27 21:02:44] no, but the idea of a pre-programmed random word isn't bad [2014-11-27 21:02:48] it would certainly be user-friendly [2014-11-27 21:03:14] MikeStir: I don't like the cost implications of any extra production operation in the factory BTW [2014-11-27 21:03:35] The OpenTRV IDs are randomly generated using local entropy on first power-up. [2014-11-27 21:03:36] there isn't any [2014-11-27 21:03:55] right, and how do you physically identify them? [2014-11-27 21:04:14] Don't be picky. [2014-11-27 21:04:17] B^> [2014-11-27 21:04:23] the usual way to do this (and it would be better if the UUIDs were globally unique) is to get the programming station to generate the ID data and print a sticky label [2014-11-27 21:04:25] lol [2014-11-27 21:04:44] Pairing, eg using the LEDs, or something else out of band would seem to be a good solutions. [2014-11-27 21:04:48] the sticky label would contain the UUID in text and barcode and could also include a friendly word [2014-11-27 21:04:50] Also harder to hack remotely. [2014-11-27 21:04:54] can't you use the mac address of the wifi modules? or...? [2014-11-27 21:05:03] what wifi modules? [2014-11-27 21:05:08] Indeed. [2014-11-27 21:05:19] the UUID _is_ the mac address essentially [2014-11-27 21:05:22] I do generate a unique 64-bit code on forst power-up. [2014-11-27 21:05:38] no you don't. you generate a _random_ 64-bit code [2014-11-27 21:05:40] No label to generate or stick on at the factory. [2014-11-27 21:06:10] At 64-bit and a reasonable attempt at real entropy gathering I'm prepared to regard it as unique. [2014-11-27 21:06:38] is it not the test spec for whatever the ATM equivalent of PCI-PTS that says that the number used to salt the crypto must never be the consecutively the same [2014-11-27 21:06:52] of course a random number has a random chance of being the same the second time [2014-11-27 21:06:57] but a counter will never be [2014-11-27 21:07:05] You are incidentally allowed to quibble with my entropy gathering. [2014-11-27 21:07:11] they reckon that's how at least one large atm fraud was executed [2014-11-27 21:07:44] Not unless you believe the whole concept of assym key generation is bogus. [2014-11-27 21:07:57] The maths is fairly clear her. [2014-11-27 21:08:26] I'll need to find the paper on that digression [2014-11-27 21:08:36] *If* my entropy collection is not bogus I can guarantee as strongly as any other empirical promise that I can make. [2014-11-27 21:09:08] As I say, feel free to attack my entropy-gathering. [2014-11-27 21:09:23] That will need careful scrutiny else I have no legs to stand on. [2014-11-27 21:09:31] I'm happy to accept that your unique ID will be unique to all intents and purposes, but the question remains - how do you identify the valve? [2014-11-27 21:09:54] by enabling them in a serial manner? [2014-11-27 21:09:55] Pairing, out of band. [2014-11-27 21:10:03] Or that. [2014-11-27 21:10:17] http://www.cl.cam.ac.uk/~rja14/Papers/unattack.pdf [2014-11-27 21:10:21] page 6 second para [2014-11-27 21:10:22] But I;d rather sort the key-exchange problem at the same time. [2014-11-27 21:10:25] section 3.1 [2014-11-27 21:10:33] And without needing Difffie-Helman over the air. [2014-11-27 21:11:14] Counters are generally pretty poor for dist algorithms or security without great care. [2014-11-27 21:11:22] if you want an asymmetric crypto then there's ECC as well which is lighter [2014-11-27 21:12:16] I think that's the point - the EMV spec intends it to be a random number, but the definition of "unpredictable" is such that you have a better chance of passing conformance testing by using a counter than you do using a rng (because of the birthday paradox) [2014-11-27 21:13:30] fhuizing: that would be pretty labour intensive. tinyhan is designed to self-organise, the idea being that you don't have to do some fiddly per-node operation [2014-11-27 21:14:22] Yes, but that is rather missing the point that it only appears better during testing by dodging the issue. I can do that dodging in other slightly better ways, but the right solution is to generate the nonces properly [2014-11-27 21:14:40] and I do seem to have a reliable entropy source with a reasonable bit rate [2014-11-27 21:15:00] DamonHD: for key exchange I was actually thinking about just using some trivial setup key and then sending a network key from the coordinator over the air [2014-11-27 21:15:19] Again, I'm not suggesting that we claim to have achieved 'security' without many eyes on all elements of our claims! [2014-11-27 21:15:20] I mean zigbee HA uses a well-known key to do the initial exchange and no one complains about that [2014-11-27 21:15:34] I do, no that you've told me. [2014-11-27 21:15:38] now [2014-11-27 21:16:02] the idea of a 4-digit pin is acceptable to users [2014-11-27 21:16:17] They still would need to know which one to use. [2014-11-27 21:16:25] print it on that label [2014-11-27 21:16:29] And they would find it less acceptable without rate limiting. [2014-11-27 21:16:34] but isn't punching in a 4 character alphanumeric id labor intensive by itself? [2014-11-27 21:16:45] I suspect that what you suggest would be too expensive. [2014-11-27 21:16:58] oh major lagspike [2014-11-27 21:16:58] expensive in what respect? [2014-11-27 21:17:02] I really do not want to have to customise each item. [2014-11-27 21:17:13] It's an extra production step or two. [2014-11-27 21:17:20] you've got to program them all so customising the eeprom contents is trivial [2014-11-27 21:17:20] But we'd have to find out. [2014-11-27 21:17:24] Not at all. [2014-11-27 21:17:34] Also printing ans sticking custom labels. [2014-11-27 21:17:40] it's automated [2014-11-27 21:17:49] Remember that OpenTRV is aiming at a factory gate price of £2. [2014-11-27 21:18:00] Every production step eats into that. [2014-11-27 21:18:10] And who knows you may be right. [2014-11-27 21:18:21] But I'd prefer a solutiion that does not add such a step. [2014-11-27 21:18:37] CEM charges for their time. It would take literally seconds to take a label from a thermal printer and stick it on the unit, or on the box [2014-11-27 21:18:44] Push that work the the user with a pairing step that they perform. [2014-11-27 21:18:51] Don't care. [2014-11-27 21:18:54] and the memory programming can be done during the firmware flash step without intervention [2014-11-27 21:19:10] I have a 60x cost reduction ahead of me. I don't want to make any hostages to fortune yet. [2014-11-27 21:19:49] I just don't want to commit to any such thing yet. All I hear tells me those costs are significant. [2014-11-27 21:19:53] But you may yet be right. [2014-11-27 21:20:06] I simply don't want to rely on that if we can find soemthing else. [2014-11-27 21:21:07] This is a hard problem I think. [2014-11-27 21:21:15] ok. so you have a unique ID but you don't know what it is, although it would register with a coordinator if you enabled it so you could find out that way, but you'll have to install each valve one by one and go back to your computer to give it a name [2014-11-27 21:21:26] We don't need to be disappointed if we can't solve it now. [2014-11-27 21:21:36] Maybe. [2014-11-27 21:21:40] then the only shared knowledge you have is that uuid, which is necesarilly sent in the clear [2014-11-27 21:21:47] Or pair it via the LED as it comes on line. [2014-11-27 21:21:54] *necessarily [2014-11-27 21:22:01] The computer is the phone which can be in your hand right? [2014-11-27 21:22:10] And see it on a Web interface *if* you care about names, [2014-11-27 21:22:34] yes you could maybe do it with a phone, but it would be a shame to mandate that [2014-11-27 21:22:37] Needs to be some sort of network gateway, [2014-11-27 21:23:00] But I *really* don't want a phone or Internet to be e necessary part of your heating system. [2014-11-27 21:23:13] My in-laws have no smart phone or Internet connections. [2014-11-27 21:23:13] avahi and a local browser [2014-11-27 21:23:36] And when Hive's servers go down you apparently can't adjust your heating. [2014-11-27 21:23:43] Not keen on that. [2014-11-27 21:23:52] but the boiler controller could do this job as well, although it would be quite awkward to do on a 2 line text lcd [2014-11-27 21:24:06] You only need the friendly names if you want to control via a fancy/smart interface. [2014-11-27 21:24:20] Really no need in the simple cases. [2014-11-27 21:24:35] Main thing would be to have boiler only respond to authorised units. [2014-11-27 21:24:38] yeah ok that's a valid point. in the case of the smart boiler controller it doesn't really need to expose the per-room data [2014-11-27 21:24:41] although it would be nice if it did [2014-11-27 21:24:44] They need not have human-known names. [2014-11-27 21:25:15] well the other thing would be to pre-register uuids, but that requires that you know them! [2014-11-27 21:25:20] Lots of nice-to-haves which we should try not to tangle ourselves up in! [2014-11-27 21:25:28] I want to avoid that. [2014-11-27 21:25:33] Did I mention pairing? B^> [2014-11-27 21:26:07] |<-- fhuizing has left irc.z.je (Ping timeout: 246 seconds) [2014-11-27 21:26:08] It's already a big nuisance for the FHT8V. [2014-11-27 21:27:28] On the pairing front we already have light sensors and LEDs for example. [2014-11-27 21:27:59] But other things might be possible such as turning the radios right down to make them more like NFC [2014-11-27 21:28:00] you could transmit the uuid via the LED to a phototransistor on the coordinator [2014-11-27 21:28:07] and using a slightly different band/carrier. [2014-11-27 21:28:31] But it would be good to have something which cannot be messed with remotely. [2014-11-27 21:28:44] And short of standing outside the window with a laser... [2014-11-27 21:28:46] -->| fhuizing (fhuizing@zje-AB3E19C8.adsl2.static.versatel.nl) has joined #opentrv [2014-11-27 21:28:58] Hi f... [2014-11-27 21:29:01] you can get NFC eeproms [2014-11-27 21:29:04] crap, I'm lagging out every time :/ [2014-11-27 21:29:19] Mike I think you showed me on of those or Stuart did. [2014-11-27 21:29:29] yeah might have been me [2014-11-27 21:29:30] Remind me what they do? [2014-11-27 21:29:52] it's a normal i2c eeprom but you can attach a coil to it and read it from a standard iso14443a reader [2014-11-27 21:29:55] or write it [2014-11-27 21:30:10] Do you think it might work for us? [2014-11-27 21:30:27] I don't want to reinvent the wheel. [2014-11-27 21:30:42] Or add a use-once-only component of course! But... [2014-11-27 21:30:51] it would work best with a phone - you could just touch the phone to the valve to share the key and forward it to the coordinator [2014-11-27 21:31:26] I was thinking of walking the valve up to the boiler unit (and/or network concentrator) to pair it. [2014-11-27 21:31:41] To make it hard to mess with remotely. [2014-11-27 21:31:50] you could put an nfc reader in the boiler unit [2014-11-27 21:32:02] but then you might as well use the led [2014-11-27 21:32:07] Could we fudge our radios to be NFC with radios down an min power? [2014-11-27 21:32:11] no [2014-11-27 21:32:15] Why not? [2014-11-27 21:32:21] NFC-like [2014-11-27 21:32:22] because it's totally different [2014-11-27 21:32:38] I simply mean talk too quietly to be overheard [2014-11-27 21:32:41] you could turn the power all the way down [2014-11-27 21:32:45] not be exactly NFC spec [2014-11-27 21:33:00] That at least as the advantage of reuse of existing components... [2014-11-27 21:33:03] it would be helpful but no gaurantee that the range will be as short as you'd like [2014-11-27 21:33:10] I understand. [2014-11-27 21:33:28] But some Vulcan button salute at the same time may be enough. [2014-11-27 21:33:40] Like Bluetooth headset pairing. [2014-11-27 21:33:40] might be worth looking at the bluetooth simple pairing schemes [2014-11-27 21:33:44] snap [2014-11-27 21:33:48] Bingo. [2014-11-27 21:33:49] particularly the "Just works" scheme [2014-11-27 21:34:35] the problem is I think there's always a window of opportunity where you can sniff the link key, and after that you are in as long as you don't miss a key update [2014-11-27 21:34:42] So, do we have a decision maybe, or a decision to investigate further, based on the radios turned down low and/or LED+photosensor and/or strange button pattern [2014-11-27 21:35:07] This doesn't have to be Fort Knox, but not vulnerable to drive-by pranking. [2014-11-27 21:35:07] I don't think we do really no [2014-11-27 21:35:20] (Says a many who refuses to use any NFC payments cards at all.) [2014-11-27 21:35:22] there's too much to think about and I'd rather get the app layer working before I start thinking about crypto [2014-11-27 21:35:32] -->| fhuizing_ (fhuizing@zje-AB3E19C8.adsl2.static.versatel.nl) has joined #opentrv [2014-11-27 21:35:33] ping [2014-11-27 21:35:33] |<-- fhuizing has left irc.z.je (Quit: Leaving) [2014-11-27 21:35:42] looks like you need to get a new isp fhuizing_ [2014-11-27 21:35:47] Well, as long we don't accidentally make it impossible. [2014-11-27 21:35:53] grmbl [2014-11-27 21:36:21] the crypto is mac layer, so the mac layer might need some tweaking, but if it breaks the app layer then we got something wrong [2014-11-27 21:37:30] If you feel happy with that I can certainly cope with any 'refactoring' risk. [2014-11-27 21:38:16] fhuizing are weletting you get a word in edgeways? [2014-11-27 21:38:34] realistically this is going to be a closed system in the short term, so I don't think there's any big problem with breaking the air interface at this stage [2014-11-27 21:38:49] |<-- fhuizing_ has left irc.z.je (Quit: Leaving) [2014-11-27 21:38:51] -->| fhuizing__ (fhuizing@zje-AB3E19C8.adsl2.static.versatel.nl) has joined #opentrv [2014-11-27 21:39:01] But I don't to have more than 2 rounds of non-interoperating OpenTRV stuff for example! [2014-11-27 21:39:03] I don't think it's my isp tbh [2014-11-27 21:39:19] google pings are 30-40ms [2014-11-27 21:39:49] But this isn't going into a shipping product probably for a year, realistically. [2014-11-27 21:39:56] So we have time to get it right and tested. [2014-11-27 21:40:46] Mike: I added a pairing note to the 'items for discussion'. [2014-11-27 21:40:54] ok [2014-11-27 21:41:32] so any more general thoughts on the tinyhan profile stuff? [2014-11-27 21:41:43] Nope. [2014-11-27 21:42:08] Can I continue on the client given the suggested scheme? [2014-11-27 21:42:09] I think we just have to get in a position to start trying it in earnest then I will find everything I hate/love about it quickly! [2014-11-27 21:42:14] I thought about having a concept of group ID and subscribed profiles, analogous to mqtt [2014-11-27 21:42:51] fhuizing__: I think what you've got sounds close to what I'm thinking so I'd keep on with that [2014-11-27 21:42:55] But we do have to think about how the ports/JSON/ new keys thing will pan out. [2014-11-27 21:43:15] Never mind MQTT. Argh! Too much for my tiny brain. [2014-11-27 21:44:17] well the application protocol is only going to define the binary protocol. if you want to borrow the approach for your json thingy then fine but you can run it on another port [2014-11-27 21:44:51] I want to keep this well defined with minimal optionality, because that is where zigbee falls down [2014-11-27 21:44:57] Maybe once we have the current unit shipping we should try and retrofit one (+ boiler unit) with a version of your stuff Mike, ignoring security to get the semantics of the rest right, JSON and binary. [2014-11-27 21:45:27] Yes, so we just need to make sure that the main combinations work. [2014-11-27 21:45:42] Then what we do when we use the stuff on e generic sensor platform. [2014-11-27 21:45:56] Ie with novel sensors and/or combinations at a leaf. [2014-11-27 21:46:08] It it copes with those, then we've won. [2014-11-27 21:46:11] I'm not even going to think about json on the air interface - it's a daft idea. The bridge will translate the binary to proper human readable json when it does the conversion to IP (whether MQTT or some HTTP based thing) [2014-11-27 21:46:31] For you yes. [2014-11-27 21:46:47] For lesser mortals it or something very like it has to remain an option. [2014-11-27 21:47:10] it's daft. you've compressed it to the point of unreadability. it's virtually binary as it is! [2014-11-27 21:47:28] So as long as that can function alongside your stuff using the same network underpinning then good. [2014-11-27 21:47:34] Nope. [2014-11-27 21:47:44] yes you can send anything you like as long as it fits in the mtu [2014-11-27 21:47:48] Shall I give you my current counter example. [2014-11-27 21:47:51] Real life one. [2014-11-27 21:47:55] ok [2014-11-27 21:48:06] I have a new sensor, RH% in this case. [2014-11-27 21:48:34] I want to start collecting the data quickly but don't have to process it yet. Will want to in real time later but not yet. [2014-11-27 21:49:26] I don't have dev time (or possibly Flash space) to rejig my (working and compact) binary format all the way up the chain, so I'm leaving that alone for now. [2014-11-27 21:49:47] And just slapping in a JSON frame for it which is now logging the data for me. [2014-11-27 21:50:24] WHEN I have decided if I like the sensor and what I want to do for units that don't have it, and how best to encode it (ie precision on the wire) [2014-11-27 21:50:36] THEN I can go back and redo my binary format once. [2014-11-27 21:51:29] So, it gets me started quickly with minimum dev effort and it already loggable and processable on the back end without having to fix all the stuff in between for an optimal format. [2014-11-27 21:51:37] It's working for me right now. [2014-11-27 21:51:48] or you could just transmit the raw values using a dimensionless type and then switch to using the RH% one from the "general environmental" profile [2014-11-27 21:52:02] But that's still a pain the the butt. [2014-11-27 21:52:15] I might as well transmit something loggable as-is. [2014-11-27 21:52:34] it doesn't gain you anything. you still have to know what the data means. just because you wrote "RH%" in some json doesn't make it any easier to deal with it at the far end [2014-11-27 21:52:39] yes it does [2014-11-27 21:52:50] I haven't had the time to do the binary rework yet. [2014-11-27 21:52:56] it could just as well have said 0x1520 where the 0x15 means RH% [2014-11-27 21:53:04] I am logging the data now to characterise how I want to us eit. [2014-11-27 21:53:19] Maybe, but I want to get some utility out of it immediately. [2014-11-27 21:53:27] you can log the data no matter how you transmit it [2014-11-27 21:53:34] it's still going to come out the end in json either way! [2014-11-27 21:53:51] Not unless I write some translation code somwehere else too. [2014-11-27 21:53:59] I really really don't have time for that now. [2014-11-27 21:54:14] So I prepare it int he format I want to log it right on the Arduino. [2014-11-27 21:54:15] if you send it as dimensionless it's going to come out of the bridge as something like "{'value1': 76} [2014-11-27 21:54:25] No further work required to log it and eyeball it. [2014-11-27 21:54:31] And process it nicely later if I want. [2014-11-27 21:54:35] One step. [2014-11-27 21:54:48] One line of code on the Arduino to add a new sensor value. [2014-11-27 21:55:18] root["H|%"] = RelHumidity.get(); [2014-11-27 21:55:23] That's it. [2014-11-27 21:55:25] yeah ok if you really just want to save the raw packets to a file, but you've got to have a bridge anyway so you can solve this problem by defining some general purpose data types now [2014-11-27 21:55:30] No code anywhere else at all. [2014-11-27 21:55:48] Don't have time to update an entire stack every time right now. [2014-11-27 21:56:04] And pragmatically a large group of users never will. [2014-11-27 21:56:10] Let's fight other battles! [2014-11-27 21:56:38] of course if you send this in binary having defined some generic type from the start then you are in exactly the same situation [2014-11-27 21:56:39] -->| Error323 (fhuizing@zje-AB3E19C8.adsl2.static.versatel.nl) has joined #opentrv [2014-11-27 21:56:49] <- fhuizing from wired connection [2014-11-27 21:56:54] But I don't know my sensor set in advance. [2014-11-27 21:57:00] tinymac_send(0x10, (uint8_t[]){0x80,32}, 2) [2014-11-27 21:57:01] done [2014-11-27 21:57:05] |<-- fhuizing__ has left irc.z.je (Quit: Leaving) [2014-11-27 21:57:13] and that doesn't even rely on a shit load of actual code hidden in libraries [2014-11-27 21:57:22] Where we do, your stuff is likely a near optimal solution for many or most cases. [2014-11-27 21:57:38] This Arduino JSON lib is 800-ish bytes total. [2014-11-27 21:57:46] Small than using xxprintf. [2014-11-27 21:57:51] Smaller [2014-11-27 21:58:33] Ignore the ID part which I will hide or at least do more elegantly: [2014-11-27 21:58:33] JsonObject<5> root; // Configure for maximum required size. [2014-11-27 21:58:33] // Compute the ID. [2014-11-27 21:58:33] char bufID[5]; [2014-11-27 21:58:33] hexDigits(eeprom_read_byte(0 + (uint8_t *)EE_START_ID), bufID+0); [2014-11-27 21:58:33] hexDigits(eeprom_read_byte(1 + (uint8_t *)EE_START_ID), bufID+2); [2014-11-27 21:58:33] bufID[4] = '\0'; [2014-11-27 21:58:33] root["@"] = bufID; [2014-11-27 21:58:33] root["T|C16"] = TemperatureC16.get(); [2014-11-27 21:58:33] #if defined(HUMIDITY_SENSOR_SUPPORT) [2014-11-27 21:58:33] root["H|%"] = RelHumidity.get(); [2014-11-27 21:58:33] #endif [2014-11-27 21:58:33] root["L"] = (uint8_t)(AmbLight.get()>>2); [2014-11-27 21:58:33] root["B|cV"] = (int)(Supply_mV.get()/10); [2014-11-27 21:58:55] Vs many many lines of knotty bit twiddling potentially. [2014-11-27 21:59:32] I have a cunning plan to make this dead dead simple for marginal programmers. [2014-11-27 21:59:42] So even I can use it. [2014-11-27 22:00:07] Meanwhile you and I get paid consultancy fees to do it right with your scheme when most features are settled. [2014-11-27 22:00:16] struct blah { uint16_t id; uint16_t temp; uint16_t rh; } = { .id=bufID, .temp=temperature.get(), .rh=rh.get()} blah; tinymac_send(1, &blah, sizeof(blah)) [2014-11-27 22:00:32] Which itself is fraught with difficulties. [2014-11-27 22:00:35] and most of that generates no actual code [2014-11-27 22:00:53] hehe [2014-11-27 22:01:03] Means I have to decode carefully (say) at my Java gateway. [2014-11-27 22:01:07] if you try to make this simple for marginal programmers you will not achieve your £2 [2014-11-27 22:01:17] Don't fight it; let it co-exist with your stuff. [2014-11-27 22:01:18] you can't afford the flash and ram [2014-11-27 22:01:23] I can. [2014-11-27 22:01:26] It's tiny. [2014-11-27 22:02:02] I was astonished how small it is. [2014-11-27 22:02:15] Vs many many lines of knotty bit twiddling potentially. [2014-11-27 22:02:18] Though I need to do some more work to make sure that I'm not fooling myself. [2014-11-27 22:02:27] root["L"] = (uint8_t)(AmbLight.get()>>2); [2014-11-27 22:02:27] root["B|cV"] = (int)(Supply_mV.get()/10); <-- you mean like this? [2014-11-27 22:02:35] No. [2014-11-27 22:03:01] It;s a world of difference x["key"] = int value. [2014-11-27 22:03:40] from working out byte and bit alignment in say C python nd Java on various possible upstream platforms. [2014-11-27 22:03:51] the bridge will translate to json [2014-11-27 22:03:57] that's always been the design [2014-11-27 22:04:00] and proper json with actual words in it [2014-11-27 22:04:13] Only if you are capable of writing code to go in the bridge [2014-11-27 22:04:23] Many potential users do not have those skills. [2014-11-27 22:04:36] do you really expect many people to be fiddling with the low level bits? [2014-11-27 22:04:39] Given the JSON escape route they never ever need to [2014-11-27 22:04:45] surely the main area of interest will be getting data out of the bridge? [2014-11-27 22:05:00] No, I expect them to slap sensors together and see JSON pop out of Xively somewhere [2014-11-27 22:05:11] which it will [2014-11-27 22:05:12] without them writing any code anywhere in fact. [2014-11-27 22:05:30] No chance whatsoever of touch code on a bridge. [2014-11-27 22:05:36] But lots of novel sensors. [2014-11-27 22:05:58] Typically supplied by third parties [2014-11-27 22:06:05] who don't supply the bridge. [2014-11-27 22:06:26] Anyway, I have some C++ tidyup to make that part of it look trivial. [2014-11-27 22:06:40] I'm still trying to find a good formulation. [2014-11-27 22:07:21] Will have to go RSN. Bed time is approaching... [2014-11-27 22:07:23] are you confident enough on the comet valves yet mike? [2014-11-27 22:07:42] no I'll be building those at the weekend hopefully [2014-11-27 22:07:44] RSN? [2014-11-27 22:07:51] Real Soon Now [2014-11-27 22:07:52] "real soon now" [2014-11-27 22:07:56] SNAP [2014-11-27 22:07:57] ah [2014-11-27 22:08:11] Like all my delivery dates... [2014-11-27 22:08:23] haha [2014-11-27 22:08:30] Right must go... [2014-11-27 22:08:36] ok, well there's no reason why the bridge couldn't have a pass-thru feature [2014-11-27 22:08:36] Anything else quickly? [2014-11-27 22:08:41] the difficulty would be in deciding where it would send it [2014-11-27 22:08:50] Mike, that's all I ask for! B^> [2014-11-27 22:09:03] Yes, definitely part of the discussion. [2014-11-27 22:09:06] maybe to nodes//passthru [2014-11-27 22:09:22] That's a neat idea. [2014-11-27 22:09:54] And without discarding the ability to add a friendly alias as suggested before. [2014-11-27 22:10:03] OK, thx. [2014-11-27 22:10:07] Must go. [2014-11-27 22:10:07] I've already experimented with a plaintext passthrough that the bridge posts to a syslog server [2014-11-27 22:10:16] that's handy - you get the sensor logs in your /var/log [2014-11-27 22:10:17] gnite [2014-11-27 22:10:18] Sorry [2014-11-27 22:10:20] o&o [2014-11-27 22:10:25] nn [2014-11-27 22:10:28] [INFO] Disconnecting from IRC. Click close again to exit now. [2014-11-27 22:10:29] [QUIT] Disconnected from irc://irc.z.je/ (irc://irc.z.je/). [[Reconnect][Reconnect to irc.z.je][reconnect]]