-->
Page 3 of 4

Re: Problem with getting "msgreturn" msg from one ESP to ano

PostPosted: Fri Jan 20, 2017 12:49 pm
by Boxey
Thank you for the replies forlotto and Electroguard! :)

forlotto; I looked into getting something off the shelf, but I sort of like pulling things apart and building things.. it keeps me out of trouble ;) As for the type of warning the system might give, I was thinking along the lines of a simple buzzer on one of the ESP outputs, but Electroguard has got my mind wandering onto the possibility of using an MP3 sound module (which sounds like it might be worth integrating to "modernise" things a bit). I was looking at some Solar PIR lights, and most seem to use the standard PIR modules that can be easily found on eBay etc, so should be easy to get to the signals I need (electronics is more my thing than networking. I once went on a networking course, and when they got to how to work out a subnet of an IP group, and got the binary ones and noughts out, I gave up and decided electronics and fewer headaches was the way to go ;) ).


Electroguard; Thank you for the long reply... it all sounds very interesting, and I'm impressed that you even thought about how I could piece things together and make it work for my senario, so thank's again.

I've been playing around with UDP and trying to get a better idea of how to make things work. I liked the "wget" system, as it seemed a bit more reliable, with very very few messages dropped ... but would still of needed some sort of checking and resending routine to make it 100% reliable, and also has the downfall of needing to know the IP's of all the nodes before it can send messages to them. UDP seems easier to just add a node to whenever you want, without knowing anything about it (and then letting it reply with things like it's IP, name and initial pin statuses).
I had a thought.. that I might try to integrate a switch on the master unit (like a pin hole switch you might find on the back of a clock or camera), which you would press to add a new node, and then it would shout out "install" to all IP's and log the details of the replying node (any nodes already on the system would have a flag set after initially sending their "new node" details , so they would automatically ignore the masters "install" shout).
From what I understand, I could set up a "udpbegin" on the master with one port address, and another "udpbegin" on the slaves with a different port address, so they are both essentially listening for messages from each other ...that way if a slave wants to talk to the master at any time, it can, and the master can talk to the slaves at any time.
One of my concerns, while playing around with UDP sending/receiving, is the amount of memory that seems to slowly drain away, until the ESP either locks up or resets. Looking around the forum, someone (it might possibly of even been yourself!), recommended having "a quiet place to rest while waiting for interrupts", and making sure all routines ended up having a "goto" back there to stop the memory stack building up ...which I hadn't thought about before, but seems logical now, so I'll work that way from now on and see if it helps.
I also wondered if there was a way to get the 1M version of BASIC on to a 4M ESP, so it had more memory to play with, but I'm thinking that if I flashed a 4M ESP with the 1M version, it may not have been set up to see the "extra" space available anyway. (as the nodes don't need any of the TFT or Graphics routines that are in the larger version).

The "Master" I had envisaged, would have a little OLED display and a buzzer ..and be easy enough to move around from room to room. I thought about having more than one "Master" to have in different rooms, but this complicates things a lot more, and most of the time the "Master" would just be in the bedroom to alert at night. (..and as my grandma always said; Simple solutions are quick solutions. She also used to say that the postman was putting birds through the letterbox! ..but that's another story!!!)

Your code seems very interesting, and it sounds like you've put a lot of time and thought into making things to work. I would, of course, be very happy to have a look at it and maybe dissect it in an attempt to make a little PIR / Lights system, and if I can, try to add some sort of non-delivery message retry thing so nothing can go astray.

I grew up with BASIC, and having both parents being school teachers, I was lucky enough to have most of the old mainframes and BBC Micros to play with when the schools upgraded... so having BASIC on an ESP has been a godsend (Thanks Mike!) and has brought a new way to pass away the winter nights ....in front of a laptop with gosubs and returns everywhere!

Thanks again for your reply and offer. It's much appreciated :)

Re: Problem with getting "msgreturn" msg from one ESP to ano

PostPosted: Fri Jan 20, 2017 1:00 pm
by bugs
Boxey wrote:<snip>
Imagine I have a long tree lined driveway, and spaced out along the driveway are 6 individual lights.
<snip>
I'm about to get some new solar powered lights with PIR's built in, and would like to separate the light and PIR signals, and put an ESP in each unit, which would tell me if the PIR had been triggered, and also let me turn each light on individually if I wanted to.


Two things occur to me -

1. Long driveway and wifi - I have managed about 20 feet away with an ESP-01 outside and the wifi access point in the house. Perhaps each ESP could pass a message to its neighbour to ensure it reaches then house?

2. Tree lined and solar powered - so shaded?
My initial ESP-01 tests were from a green house using a 750mAh Li-Po battery and an added reset wire so the ESP could sleep for 10 minutes. The ESP takes a LOT of current and the battery was discharged in about 4 days.

Electroguard - would be interested to see what you achieved - especially if a modular approach...

Re: Problem with getting "msgreturn" msg from one ESP to ano

PostPosted: Sat Jan 21, 2017 8:46 am
by Electroguard
Boxey, I had to laugh when you said you'd grown up with old BBC Micros to play with, cos they weren't even manufactured when I ordered mine... I had to pre-order, then wait 6 months for it to arrive.

Addressing Bugs very relevent range comments first, I currently have a dedicated 'systems' wifi access point mounted under the corner of the hanger roof with clear line-of-sight. It purposely doesn't have internet access or any other network traffic, although eventually I may have a serially-connected internet bridge if I have need of external access. Until then, I'll be using a local i2c RTC module to keep time and date. The AP gives complete Esp_Basic coverage of my 4000 sq m property during walk tests, but I've already got a new 4-aerial wifi router to replace it with.

Mounting external 2.4Gb aerials on quadcopters and their transmitters is a simple way of extending quadcopter wifi control range from about 30 m to hundreds of metres. So my plan is to mount the new router inside the hanger but replace some of its sticky-up aerials with externally mounted aerials, including a high-gain directional beam I have waiting ready.

And Forlotto,while on the subject of extending range, similarly for the quadcopters 5.8Gb FPV video stream as well... and there are several cheap (<£50) video overlay systems available which overlay GPS info plus things like battery voltage and all sorts of other stuff on top of the camera video stream... which makes them useful for re-purposing for other things.

Boxey, I expect you could utilise some of your existing driveway signal wiring to use as supplementary low-voltage supplies if needed, if you didn't want to use a wifi beam aerial range extender, you might even be able to power one or more (low volts) remote wifi repeaters if needed.


I tried a bunch of different PIR solar lights before I found one that was suitable. I wanted it to use 3.7v lipo battery rather than 1.2v rechargables, and to have room to add a second (or bigger) lipo battery if necessary (there are many different sized quadcopter lipo's available), plus have room for an ESP-12 with relay and LDR.
Another important criteria was the ability to fit a couple of small magnets for quick-change mounting (nylon countersunk M4 screws and nuts, and magnets with countersunk holes, are all cheap from ebay). This allows me to readily 'plop' them around the sides of my metal hanger, onto the external mailbox, and onto metal screw or nail fixings on wooden posts etc. And if you can relate Boxeys Driveway monitoring requirements to over-nighting in a camper, magnetic solar PIRs allow simple 'plop-on' security around the perimeter of the camper giving external lights warning to anyone approaching too close, plus internal audible alerts on the mobile phone (and even video streams of strategically placed miniature quadcopter FPV cameras if wished).
The solar PIRs I settled on were cheap but not durable, so I bought 40 of them and added magnets to all (use duct tape over the magnets to prevent them from scratching paintwork), which gives me plenty of ready-to-use shelf spares. They are all physically the same, so just need remotely naming as appropriate when used). In my case I require most of them as security sensors rather than lights, therefore I snipped of half of the LEDs on some to reduce battery drain, but for lighting purposes you'd probably be better fitting a larger lipo battery.

So I have the hardware pretty much already sorted, it just remains to add fault tolerance to the firmware... which was nice and stable until overloaded to insanity by my attempts to add re-transmission of queued un-acknowledged previous message transmissions.
A brief explanation of that... the EasyNet path had been long, and forced many compromises from imposed code dieting before lean enough to maintain stable and consistent operation. This was proved for dev purposes by adding a memory meter facility, the original intention being to monitor for memory leakage then trigger a controlled Esp reboot before it hung. But I was eventually able to eliminate all causes of memory leak, and found that things remained stable indefinitely (for days anyway) during my testing... and this was in spite of the additional overheads taken up by using the memory meter. Similarly for the debug message window mechanism, which used valuable resources, but was invaluable for debugging.
So the EasyNet messaging was working fine... I just needed to add some fault-tolerance.
I thought using arrays would use more resources than adding transmitted messages to a 'queue' string then deleting them from the queue string when an ACKnowledgement was received.
So the plan was to add an optional 'Quality of Service' qos=(retries) facility to messages, whereby any transmitted message containing "qos=" would be added to the transmitted queue string which would be periodically checked for remaining un-acknowledged transmitted messages. Periodically, any remaining un-ACKed messages would have their qos= value decremented and the message be re-transmitted, until either deleted due to receiving an ACK of the message, or else its qos=0 causing it to be ditched.
A necessary complication was to add queued message IDs (qid=value) to allow incoming acknowledgements to be matched up to their original queued message for deletion - bearing in mind that any communications problem might cause multiple failed messages intended for multiple different targets, with no way of knowing what order acknowledgements might be received. I used an elapsed timer, whose value was used to include a unique qid=elapsed_time value to each transmitted message.

But it was a all a step too far, and strange symptoms caused by lack-of-memory corruptions occured which prevented me developing the required message queue management facilities.
And that's the current state of the script... it was basically working until attempts to add message queueing and retry facility pushed it over the edge to insanity. It's not difficult to omit the qos= and qid= options and return back to simple networking without fault-tolerance, but to do so is a bit like flying a jumbo jet on autopilot... easy when you know what to do and why, but you still need to know how to take off first. And there are definitely a few things you need to know to get the EasyNet jumbo flying, cos although flying it is very easy (it almost flies itself), you still need to know what controls to turn on.

This has already taken me an age, but I'll add a quick overview of things to keep you going until I can get time to straighten the script out from its last prodding and add some simple flying instructions for it

You don't need to know any individual network addresses, because all message targeting is done by name.
Each node is given a unique node name, preferably something self-explanatory like DRIVEPIR3 and HOUSE1. Optionally each node can also be given a Group name (such as SOLARLIGHT) and System name (such as DRIVEWAY). UDP messages are broadcast to all nodes on the subnet, but messages can be sent for the attention of named targets by including the option "target=(Nodename or Groupname or Systemname or IPaddress)". Alternatively the same can be achieved with the shorter "t={targetname)" version for dev convenience.

You won't even need a 'Master' as such, because all EasyNet nodes can be created equal using the same identically configured script, and their individual behaviour will be dependent on their assigned name which will dictate what broadcast command messages they respond to.
Any triggered driveway sensor nodes for example, including the beambreak, could trigger a ding dong connected to HOUSE1 Relay2 by sending the identical UDP broadcast RELAY2 CYCLEON target=HOUSE1, causing a momentary ding dong bell to go off. If you sent appropriate triggering location info you could light specific location LEDs on your monitor panel, or play a corresponding mp3 announcement file.
The HOUSE1 gpio00 button might be used to UDP broadcast RELAY2 CYCLEON target=SOLARLIGHT to turn on the lights of all driveway nodes belonging to the SOLARLIGHT group for a predetermined duration.

So when you think about it, the same script configuration could be used for all of your nodes - and a great advantage of that is you can have ready-to-use replacements standing by ready to swap out and rename as approprtiate.

You could add your own fault tolerance, but I doubt it's even necessary for your purposes... it's unlikely that different sensors nodes will be triggered to broadcast alerts at the same time, and it could easily be countered anyway by transmitting alerts twice in succession with a small random delay in between, or a delay dependent on the last byte of their IP address times 10 or 100.
Got to go. I think that should give you plenty to be mulling over for the moment.

Re: Problem with getting "msgreturn" msg from one ESP to ano

PostPosted: Sun Jan 22, 2017 1:15 am
by forlotto
Why not using EG's setup do the following.

A master sender outside and inside.

A master Reciever Inside and outside

A routine to set flags when messages are send and clear them.

How can this be accomplished?

Someone shows up it trips pir.
1 From here a routine sets a flag (registers to 1)
2 Then the next step is to set a delay for 30 seconds (you can fine tune the delay)
3 After the delay check if flag is clear (registers to 0).
4 If they are than do nothing.
5 If not clear resend message.
6 Now during the delay in step 2 Master Sender Outside forwards the message to Master Reciever Inside.
7 Master Receiver Inside Repeats signals to inside slave buzzers.
8 Slave buzzers buzz and send clear flag signal to Master Sender Inside
9 Master Reciever outside Sends Message to slave PIR to clear flag.

You could trim it down and have 1 Master Sender/Reciever Inside and out but you get the picture.


That will distribute to all rooms. This way you could integrate your oled and button into say a lamp or something in each room.

Should be possible I used a common instance of 0 or 1 but it could be ON or OFF or whatever you wish the register to be could even set other registers for sending when the buzzer has stopped you WILL have to play with timing and add whatever you need. But flags are the way to go here to ensure data transmission has made its way to where it needs to be. You could have flags for each type of message as well.
flag 1 could mean buzzer sound flag 2 could mean lights on flag 3 could mean buzzer sound and turn lights on. This way it will make it easier to make a resend routine. Just remember the flags can be as many registers as you want them to be this is the way around registers you can have as many registers as you want with flags but you must define them and have complex ways of clearing them and adding to them.

Anyhow hope this helps.