-->
Page 1 of 1

Reading a Bin file and parsing data

PostPosted: Mon Apr 17, 2023 10:10 pm
by Jono72
Hi All

Just wondered if anyone can point me in the right direction, I don't think I'm far off but some advice from a seasoned developer would be helpful, So my project I finally managed to Ethernet WebSocket working with my ESP8266 project, and this is the final hurdle. I have a page to send firmware updates from the browser, I alread have webserver implemented and because im using Ethernet I cannot use the HTTPupdater library, so I decided to use the Updater.h library native to the ESPcore.
So I get my .bin file sending from the browser to the ESP, of course this just reads and reads until the cpu does a WDT reset.

I have the function for the updater setup and this must get the file in 128byte chucks until finished then perform an ESP reset.
So in my serial monitor thats reading the String data recived I get the following when I send firmware file

Code: Select all 

Content-Disposition: form-data; name="firmware"; filename="MQTT_POESEN_V3_TS.ino.d1_mini.bin"
Content-Type: application/octet-stream

�  @\� @� @
 �? �?  K@  �$@  ������¬���?� ����� @



I have tried to modify some code that catches the firmware file, but this is based this code I found as below.

Code: Select all
// If file is reachable, start downloading
  if(resp > 0){
      // get length of document (is -1 when Server sends no Content-Length header)
      totalLength = client.getSize();
      // transfer to local variable
      int len = totalLength;
      // this is required to start firmware update process
      Update.begin(UPDATE_SIZE_UNKNOWN);
      Serial.printf("FW Size: %u\n",totalLength);
      // create buffer for read
      uint8_t buff[128] = { 0 };
      // get tcp stream
      WiFiClient * stream = client.getStreamPtr();
      // read all data from server
      Serial.println("Updating firmware...");
      while(client.connected() && (len > 0 || len == -1)) {
           // get available data size
           size_t size = stream->available();
           if(size) {
              // read up to 128 byte
              int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
              // pass to function
              updateFirmware(buff, c);
              if(len > 0) {
                 len -= c;
              }
           }
           delay(1);
      }
  }


Ethernet client does not have client get stream, and I am trying to work out how to get the file and its length so as I can parse data to the updater function, this is what I have so far but I realize this is wrong any pointers please, at this point my ethClient in the ESP is already running!

Code: Select all
if (strstr(req, "firmware"))
{ //Get Firmware File Stream

int firmdat = ethClient.read(); //var to catch the bin data stream, not sure if this has to be an array


// Get file, just to check if each reachable
//int resp = firmdat;//client.GET();
Serial.print("Response: ");
Serial.println(firmdat);
// If file is reachable, start downloading
    //== 200
if(firmdat > 0 ){
// get length of document (is -1 when Server sends no Content-Length header)
totalLength = ethClient.getSize();
// transfer to local variable
int len = totalLength;
// this is required to start firmware update process
Update.begin();//UPDATE_SIZE_UNKNOWN
Serial.printf("FW Size: %u\n",totalLength);
// create buffer for read
uint8_t buff[128] = { 0 };
// get tcp stream
ethClient * stream = ethClient.getStreamPtr();
// read all data from server
Serial.println("Updating firmware...");
while(ethClient.connected() && (len > 0 || len == -1)) {
// get available data size
size_t size = stream->available();
if(size) {
// read up to 128 byte
int c = stream->readBytes(buff, ((size > sizeof(buff)) ? sizeof(buff) : size));
// pass to function
updateFirmware(buff, c);
if(len > 0) {
len -= c;
}
}
delay(1);
}
}else{
Serial.println("Cannot download firmware file.Something is very wrong you messed up!!");
}




This code is in my web command loop and strstr catches the data as received.

and this is the updater function itself.

Code: Select all// Function to update firmware incrementally
// Buffer is declared to be 128 so chunks of 128 bytes
// from firmware is written to device until server closes
void updateFirmware(uint8_t *data, size_t len){
  Update.write(data, len);
  currentLength += len;
  // Print dots while waiting for update to finish
  Serial.print('.');
  // if current length of written firmware is not equal to total firmware size, repeat
  if(currentLength != totalLength) return;
  Update.end(true);
  Serial.printf("\nUpdate Success, Total Size: %u\nRebooting...\n", currentLength);
  // Restart ESP to see changes
  ESP.restart();
}



Many Thanks

Jono

Re: Reading a Bin file and parsing data

PostPosted: Thu Apr 20, 2023 12:51 pm
by rooppoorali
Hello, is there any baud rate mismatch? I mean, this type of garbage value usually appears when the baud rate declared in your code is different from the one you select when opening the serial monitor. You have not copy-pasted your setup function in the snippet. So, I can't see your baud rate. Please check.

Re: Reading a Bin file and parsing data

PostPosted: Fri Apr 21, 2023 2:03 am
by Jono72
Hi thanks for the reply, no there is no mismatch, what your seeing is the binary stream data being read as it comes in for the browser, the web page sends an Ajax send, the data is the buffer in my web function that reads data from the Ethernet connection.