-->
Page 1 of 3

InqPortal - A RAD, IoT, Application, Web Server

PostPosted: Mon Mar 07, 2022 10:28 am
by Inq720
I would like to introduce a new Arduino IDE library for Rapid Development of IoT type applications for the ESP8266.  I should mention first, this library has been around since 2016.  In fact, introduce means, I now feel good enough about it to release this Version 5.0 to anyone.  Even... the hardest audience of them all... this community.

Question of cost - It's free and royalty free for you selling a product using it.


Release5.png


Why Another Web Server
I'm a retired engineer (10 years) and retired professional software developer (25 years).  I still like to develop software... far more now than when I worked for a large corporation.  I also needed a little more brain challenge than a morning Sudoku.  Besides developing, I also volunteer to teach Arduino electronics and programming at my local community college and my local library.  My students range from pre-teens to senior citizens.  Almost without exception, all are keenly interested in getting their hardware widget working.  They don't want to mess with wiring displays and pots to control their projects.  They neither have the skills or patience to build a web server out of the generalized libraries we currently have access to here.  At home, they have smart "this's and that's" and want something just like them... but of their own creation.  They want to access it from every device and anywhere.  

Fundamentals of InqPortal
There are plenty of fine libraries out there that I know of... I am most familiar with the core's ESP8266WiFi and ESPAsyncWebServer libraries.  In fact, early versions of InqPortal used those under the covers.  At the lowest level, this version 5 is a blank-sheet design of just about every subsystem including:
  • A web server based on Espressif core methods and lwIP.
  • A WebSocket with a deep dynamic buffer and metering.
  • A new EEPROM equivalent that can have multiple instances, variable sector usage and flash memory wear leveling.
  • A new file system that also has variable sector usage and flash memory wear leveling

Value Added of Fundamentals
Those things I mainly did because it was just a fun challenge I set for myself.  I am uncertain of value added as I do not know if I had fully optimized those other libraries before benchmarking them.  But I can state the performances of some of this and you can be the judge if it is comparable.

  • My test app with InqPortal v4 based on ESPAsyncWebServer used 470KB.   Version 5 has many more features, yet the same app is now 394KB.  It now cleanly fits on a 1MB ESP-01, can do OTA upgrades and still have a 196KB file system.
  • Dragging and dropping 1.8 MB of files from Windows takes about 15 seconds.
  • In an acid test using 9 browser clients displaying a web page served by the ESP8266, then immediately re-requesting the same page were able to clock over a 100 hour period 180 GB of data transferred.
  • The acid WebSocket test using four clients is able to run for 100 hour (my acceptance test) using 10 ms intervals at a rate of 32 KB/sec.  Using 1 ms intervals, it will pump out 135 KB/sec.   However, after several hours and several GB, it randomly goes belly up.  I have a thread for this issue on this site trying to get Advanced advice.  I am having troubles with the reference I found that sounds like the issue.  I don't know what it means and if there is something I can do at my level to compensate for it.
  • I should note - The ESP8266 doesn't really have a concept of CPU utilization like a PC since it always runs at 100% all the time.  InqPortal uses the rate of loop() method calls as a gauge of processor load.  This is one of the InqPortal Admin's metrics.  Nominally, under no load, this loop frequency runs around 130 kHz using the standard 80 MHz CPU frequency.  The above tests under maximum WiFi throughput still hover the loop frequency around 100 kHz.  Under version 4, this frequency tended to run around 30 kHz.

The Real Meat and Potatoes
My students don't care about those fundamentals... They may not even know they're using such things.  They expect them to JUST WORK like their cell phone!  The real value added for them is not having to mess with any of those "petty logistical" things.  They want to fire up the Arduino IDE, cherry pick several libraries, load an example and press the compile key.  They want to wire up their project, add their sensor, servo, light... etc, add this library and instantly control it from their development machine or their phone.

InqPortal lets them have that option...Three lines of code gets you to the Admin GUI shown above. Another line per variable will let you see and/or change your Sketch variables. Three more mouse clicks and it will generate a starter page using your Sketch Variables that you can take and beautify using standard web development techniques to your heart's content.

Even for Experts
I use it all the time now and find value even beyond the quick and pretty UI of the project.  Its ability to have the equivalent of debugger Watch Variables and togglable, remote logging alone keeps me from using the plain Serial.printf() anymore.  I can leave the logging in and turn it on/off as needed.  The ability to correlate project variables visually in conjunction with system performance parameters like voltage, loop (Hz) and memory usage highlights many potential bugs without explicitly having to Serial.printf them.  A ten second perusal of a three-day histogram might just save you countless hours in a fruitless deep dive of megabytes of your Serial monitor data.

If any of this sounds of use to you, please give it a try and give me back some constructive criticism.  There is always room for improvement.  There are many things planned for future versions.  These were the ones most critical for my students.  

It's my first attempt at using GitHub and adding a library to Arduino, but... I've just published InqPortal version 5.0 and it is available in your Arduino IDE, Library Manager.

Since you all are experts, you might have the tendency to avoid documentation... but if you actually want to compile it, you'll need to heed one thing about patching the Core ESP8266 library... https://inqonthat.com/arduino-ide-configuration-for-windows/#Patch

The cross-referenced documentation, examples, tutorial, quick-start guide are all available on-line at:
Release Notice - https://inqonthat.com/inqportal-5-0-iot-server-library-released-catch-it-now/
InqPortal Page - https://inqonthat.com/inqportal/
Quick Start Guide - https://inqonthat.com/inqportal-tutorialbrusk/
Tutorial - https://inqonthat.com/inqportal-use/

The InqPortal Admin also has context sensitive help that jumps to the pertinent place on the website just like the Arduino IDE.  The documentation is still rough around the edges as I'm still upgrading it from version 4 verbiage and adding the context sensitive pages.  So some is still TBD.

Thank you for your time and consideration.
Inq

Re: InqPortal - A RAD, IoT, Application, Web Server

PostPosted: Tue Mar 08, 2022 3:59 am
by QuickFix
I'm intrigued; this really looks like a great project: thanks for sharing. :D

Re: InqPortal - A RAD, IoT, Application, Web Server

PostPosted: Tue Mar 15, 2022 7:02 am
by EduardoEG
It looks like a very beautiful and useful project!!!

Version 5.2.0 Is Available

PostPosted: Tue Mar 15, 2022 10:19 am
by Inq720
Here is the more verbose version https://inqonthat.com/inqportal-5-2-rocket-sockets/, but you get the CliffsNotes version...

Improvements in this upgrade
[list=1]
[*] Upgrading of the WebSocket communications between client and server. In 5.0.0, it was topping out around 32 KB/second with occasional data loss of about 0.1%. This 5.2.0 version is still going strong in an abusive test. It is delivering 50 byte messages at a rate of 1000Hz to four clients. Total server output is slightly over 200 KB/second. With no data losses. (Issue https://www.esp8266.com/viewtopic.php?p=93809#p93809 still exists)
[*] Added functionality to permit publishRW set() methods to be used to trigger server-side events without requiring a variable to be defined.
[*] Added ButtonClickEvent example with server INO and client HTML page to illustrate usage. On-line documentation is TBD.
[*] Improved UI experience of client handling during configuration and switching device (that browser client is on) from direct SoftAP connection (to do the configuration) over to the same router configured and connecting via StationAP.

The significant WebSocket performance increase has me re-evaluating adding binary data transport through the WebSockets. It is being deferred for higher priority items. Binary would have been more fiddly for the developer and the added information rate does not appear to be necessary. Need a use case where higher information rate is necessary.