ESP8266 Community Forum

- Wed Aug 17, 2016 8:09 am #53137

Should I expect faster download of five 50KB files using ESPAsyncWebServer than a single 250KB file using ESP8266WebServer ?

I presume, Yes, but I didn't tried it myself ...

- Wed Aug 17, 2016 4:22 pm #53179 I ran into this problem a while ago then we switched to UDP so the original code is gone. Also it used plain TCP versus HTTP but I doubt this makes any difference.

I do not know why a write() size of 2*MSS versus 1*MSS makes a difference. On the Linux IP stack this change would not affect throughput in such a major way.

The problem can be recreated using the FSBrowser example.

Run File | Examples | ESP8266WebServer | FSBrowser. Modify the SSID and password as needed. Add a 1 Mbyte file named bigfile.dat and upload it to SPIFFS. The following shows it takes 52 seconds to download the file. 192.168.1.189 is the IP address of the ESP running FSBrowser.

Code: Select all$ wget http://192.168.1.189/bigfile.dat
--2016-08-17 09:27:45--  http://192.168.1.189/bigfile.dat
Connecting to 192.168.1.189:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576 (1.0M) [text/plain]
Saving to: ‘bigfile.dat’

100%[======================================>] 1,048,576   18.0KB/s   in 54s   

2016-08-17 09:28:39 (19.0 KB/s) - ‘bigfile.dat’ saved [1048576/1048576]


Doubling the buffer size reduces the download time from 54 s to 1.9s.

Code: Select all$ wget http://192.168.1.189/bigfile.dat
--2016-08-17 09:33:02--  http://192.168.1.189/bigfile.dat
Connecting to 192.168.1.189:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1048576 (1.0M) [text/plain]
Saving to: ‘bigfile.dat’

100%[======================================>] 1,048,576    532KB/s   in 1.9s   

2016-08-17 09:33:04 (532 KB/s) - ‘bigfile.dat’ saved [1048576/1048576]


The change is to ESPWebServer.h.

Code: Select alltemplate<typename T> size_t streamFile(T &file, const String& contentType){
  setContentLength(file.size());
  if (String(file.name()).endsWith(".gz") &&
      contentType != "application/x-gzip" &&
      contentType != "application/octet-stream"){
    sendHeader("Content-Encoding", "gzip");
  }
  send(200, contentType, "");
  //return _currentClient.write(file, HTTP_DOWNLOAD_UNIT_SIZE); // Original code 1*MSS
  return _currentClient.write(file, 2*HTTP_DOWNLOAD_UNIT_SIZE); // 2*MSS
}


- Thu Aug 18, 2016 3:38 pm #53246

Doubling the buffer size reduces the download time from 54 s to 1.9s.

OMG 27x faster ! That's what I call an improvement.
My first reaction would be to think a cache gets used on the second call, but as you're using wget, I guess not.
According to my quick reading of the write() implementation, it makes no sense that doubling the unit size increases the speed, and even less that it increases it by more than a factor 2.
I have to reproduce that and fiddle with the values to try and understand what the magic is.

That looks really promising anyway, particularly considering that downloading files in HTTP from SPIFFS is exactly my final use case. Thanks very much for the hint !

Kind regards,

Vicne

- Fri Aug 19, 2016 8:49 am #53293

The problem can be recreated using the FSBrowser example.

Hi, bbx10node,

Edit : I first didn't see any improvement due to a mistake of mine.
Please note the numbers of the "original version" below and compare with next post

I started with the unchanged FSBrowser example and uploaded 3 pictures to the SPIFFS, named test100K.jpg, test500K.jpg and test1MB.jpg (with the corresponding approximate sizes).
I than ran a script that performs 10 times a download of each of those pictures using wget, and checks after each download that the file is binary identical to the original file (it always was).
Finally, I changed the source file according to your instructions (I guess you meant ESP8266WebServer.h not ESPWebServer.h) and performed the same tests in the same situation as the first one (same router and same positions of ESP and PC relative to it).

Results with standard "_currentClient.write(file, HTTP_DOWNLOAD_UNIT_SIZE);" :
2016-08-19 13:41:21 (25,3 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:41:47 (19,7 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:42:32 (23,0 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:42:34 (52,5 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:42:56 (22,7 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:43:40 (23,4 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:43:45 (25,2 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:44:08 (22,5 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:44:52 (23,4 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:44:56 (25,2 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:45:18 (22,9 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:46:03 (23,3 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:46:07 (25,3 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:46:24 (29,9 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:47:08 (23,5 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:47:12 (25,1 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:47:34 (22,9 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:48:19 (23,1 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:48:23 (25,1 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:48:45 (22,8 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:49:30 (23,3 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:49:34 (24,8 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:49:57 (22,6 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:50:44 (23,5 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:50:48 (25,2 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:51:10 (22,8 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:51:55 (23,2 KB/s) - `test1MB.jpg' saved [1046750/1046750]
2016-08-19 13:51:59 (25,2 KB/s) - `test100K.jpg' saved [98828/98828]
2016-08-19 13:52:21 (22,9 KB/s) - `test500K.jpg' saved [512705/512705]
2016-08-19 13:53:05 (23,5 KB/s) - `test1MB.jpg' saved [1046750/1046750]

<snip>

See next post for results with modified version

Last edited by Vicne on Fri Aug 19, 2016 9:52 am, edited 1 time in total.