Application Size in Flash

You can chat about native SDK questions and issues here.
 Reply

User avatar

Re: Application Size in Flash

By Inq720
- Thu Nov 18, 2021 5:59 am #92868 Thank you for replying. Sounds like you can lead me to where I need to be.
I've been using the Arduino IDE for about ten years now. I'm guessing the flash partitioning is taken care of for me by the IDE via ...
ide.png

As of last week I found there is finally a non-Linux solution via Visual Studio Code and PlatformIO to build on Windows. So, to answer your question, I am using the NonOS version... real Noob level. The only method I've had to add to my user_main.c so far, is...
Code: Select alluint32 ICACHE_FLASH_ATTR user_rf_cal_sector_set(void)
{
    enum flash_size_map size_map = system_get_flash_size_map();
    uint32 rf_cal_sec = 0;

    switch (size_map) {
        case FLASH_SIZE_4M_MAP_256_256:
            rf_cal_sec = 128 - 5;
            break;

        case FLASH_SIZE_8M_MAP_512_512:
            rf_cal_sec = 256 - 5;
            break;

        case FLASH_SIZE_16M_MAP_512_512:
        case FLASH_SIZE_16M_MAP_1024_1024:
            rf_cal_sec = 512 - 5;
            break;

        case FLASH_SIZE_32M_MAP_512_512:
        case FLASH_SIZE_32M_MAP_1024_1024:
            rf_cal_sec = 1024 - 5;
            break;

        case FLASH_SIZE_64M_MAP_1024_1024:
            rf_cal_sec = 2048 - 5;
            break;
        case FLASH_SIZE_128M_MAP_1024_1024:
            rf_cal_sec = 4096 - 5;
            break;
        default:
            rf_cal_sec = 0;
            break;
    }
    return rf_cal_sec;
}


I understand the first sector and last four sectors are reserved by Espressif lower-end usage. This routine simply defines where "I" want Espressif to put more Espressif controlled data calibrating the Radio RF settings. This example code puts it in the 5th sector from the end. I do also wonder where the Arduino IDE puts this??? I know they typically place the EEPROM library sector in this 5th from the last sector. Anyway... besides these six sectors, the rest is fair-game for me to place program, file systems and anything else I want.

By your last message, I should (be able to) define everything explicitly. I've "now" dug around in the folders in the projects I've got working and I don't see anything like a make file or a configuration file. I guess I must be using some default settings that PlatformIO has made. Since I'm using PlatformIO, I don't see an explicit Make file. All I've modified so far is the platformio.ini file.
Code: Select all[env:d1_mini]
platform = espressif8266
board = d1_mini
framework = esp8266-nonos-sdk
build_flags = -I include
monitor_speed = 115200
upload_speed = 921600


Thank you for your help.
You do not have the required permissions to view the files attached to this post.
User avatar

Re: Application Size in Flash

By Inq720
- Sun Nov 21, 2021 7:53 am #92879 Trying to dig into this, I found this page: https://github.com/espressif/ESP8266_NONOS_SDK

At the bottom, it gives some very simple instructions.
  • Add user_pre_init()
  • and use system_partition_table_regist() to define the table.
Unfortunately, this is very discouraging.
  1. I added the user_pre_init() method to my user_main.c along with the user_int() method expecting it to be called. It did compile and run, but it was never called.
  2. the system_partition_table_regist() method is described in the latest manual, however, it does not exist in the SDK code base and the compiler barfs on it.

I'm not sure where to go from here.
User avatar

Re: Application Size in Flash

By davydnorris
- Sun Nov 21, 2021 5:54 pm #92882 The moment you were talking about the cal_sector_set function I knew you were on an old 2.x SDK. The only control over your layout that you had in the old SDK (and I am not 100% sure if Arduino is still using 2.x or not) was when you actually flashed your code, and the rf_cal_sector_set function was needed to tell the SDK where it could store its private Wifi data.

In the 3.x SDK they completely changed this and created a partition table - this move really messed people up for a while, but it's quite neat when you convert. There are several partitions that you *must* define for the SDK itself, but then you have the freedom to create your own and then you can reference them elsewhere in your code.

This is part of my start up code in user_main.c. I have defined three extra partitions - a couple of SSL certificate areas, and a flash object storage area:

Code: Select all#ifndef SPI_FLASH_SIZE_MAP
#define SPI_FLASH_SIZE_MAP 0
#endif

#if ((SPI_FLASH_SIZE_MAP == 0) || (SPI_FLASH_SIZE_MAP == 1))
#define SYSTEM_PARTITION_OTA_SIZE              0x0
#define SYSTEM_PARTITION_OTA_2_ADDR            0x0
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x0
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x0
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x0
#define FLASH_STORE_ADDR                       0x0
#define GEO_SSL_SERVER_CA_ADDR                 0x0
#define IOTF_SSL_SERVER_CA_ADDR                0x0
#error "The flash map is not supported"
#elif (SPI_FLASH_SIZE_MAP == 2)
#define SYSTEM_PARTITION_OTA_SIZE              0x07a000
#define SYSTEM_PARTITION_OTA_2_ADDR            0x081000
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x0fb000
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x0fc000
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x0fd000
#define FLASH_STORE_ADDR                       0x07b000
#define GEO_SSL_SERVER_CA_ADDR                 0x07e000
#define IOTF_SSL_SERVER_CA_ADDR                0x07f000
#elif (SPI_FLASH_SIZE_MAP == 3)
#define SYSTEM_PARTITION_OTA_SIZE              0x07a000
#define SYSTEM_PARTITION_OTA_2_ADDR            0x081000
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x1fb000
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x1fc000
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x1fd000
#define FLASH_STORE_ADDR                       0x07b000
#define GEO_SSL_SERVER_CA_ADDR                 0x07e000
#define IOTF_SSL_SERVER_CA_ADDR                0x07f000
#elif (SPI_FLASH_SIZE_MAP == 4)
#define SYSTEM_PARTITION_OTA_SIZE              0x07a000
#define SYSTEM_PARTITION_OTA_2_ADDR            0x081000
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x3fb000
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x3fc000
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x3fd000
#define FLASH_STORE_ADDR                       0x07b000
#define GEO_SSL_SERVER_CA_ADDR                 0x07e000
#define IOTF_SSL_SERVER_CA_ADDR                0x07f000
#elif (SPI_FLASH_SIZE_MAP == 5)
#define SYSTEM_PARTITION_OTA_SIZE              0x0fa000
#define SYSTEM_PARTITION_OTA_2_ADDR            0x101000
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x1fb000
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x1fc000
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x1fd000
#define FLASH_STORE_ADDR                       0x0fb000
#define GEO_SSL_SERVER_CA_ADDR                 0x0fe000
#define IOTF_SSL_SERVER_CA_ADDR                0x0ff000
#elif (SPI_FLASH_SIZE_MAP == 6)
#define SYSTEM_PARTITION_OTA_SIZE              0x0fa000
#define SYSTEM_PARTITION_OTA_2_ADDR            0x101000
#define SYSTEM_PARTITION_RF_CAL_ADDR           0x3fb000
#define SYSTEM_PARTITION_PHY_DATA_ADDR         0x3fc000
#define SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR 0x3fd000
#define FLASH_STORE_ADDR                       0x0fb000
#define GEO_SSL_SERVER_CA_ADDR                 0x0fe000
#define IOTF_SSL_SERVER_CA_ADDR                0x0ff000
#else
#error "The flash map is not supported"
#endif

static const partition_item_t at_partition_table[] = {
    { SYSTEM_PARTITION_BOOTLOADER,       0x0,                                    0x1000},
    { SYSTEM_PARTITION_OTA_1,            0x1000,                                 SYSTEM_PARTITION_OTA_SIZE},
    { SYSTEM_PARTITION_OTA_2,            SYSTEM_PARTITION_OTA_2_ADDR,            SYSTEM_PARTITION_OTA_SIZE},
    { SYSTEM_PARTITION_RF_CAL,           SYSTEM_PARTITION_RF_CAL_ADDR,           0x1000},
    { SYSTEM_PARTITION_PHY_DATA,         SYSTEM_PARTITION_PHY_DATA_ADDR,         0x1000},
    { SYSTEM_PARTITION_SYSTEM_PARAMETER, SYSTEM_PARTITION_SYSTEM_PARAMETER_ADDR, 0x3000},
    { FLASH_STORE,                       FLASH_STORE_ADDR,                       FLASH_STORE_SIZE},
    { GEO_SSL_SERVER_CA,                 GEO_SSL_SERVER_CA_ADDR,                 0x1000},
    { IOTF_SSL_SERVER_CA,                IOTF_SSL_SERVER_CA_ADDR,                0x1000}
};

void ICACHE_FLASH_ATTR user_pre_init(void) {
    system_phy_set_powerup_option(3);
    if(!system_partition_table_regist(at_partition_table, sizeof(at_partition_table)/sizeof(at_partition_table[0]),SPI_FLASH_SIZE_MAP)) {
        INFO("USER", "system_partition_table_regist fail");
        while(1);
    }
}


Elsewhere, I then have code like the following:

Code: Select all    partition_item_t partition_item;

    if (!system_partition_get_item(FLASH_STORE, &partition_item)) {
        INFO("STORAGE", "Get partition information fail");
        return;
    }


You could try to see if the Arduino version of the SDK is 3.x by calling the function above with one of the SDK defined partition types
User avatar

Re: Application Size in Flash

By Inq720
- Mon Nov 22, 2021 11:21 am #92885
davydnorris wrote:You could try to see if the Arduino version of the SDK is 3.x by calling the function above with one of the SDK defined partition types


Again, thank you for your reply and I don't want to abuse your help...

None of my current environments recognized the method system_partition_get_item().
Using the method system_get_sdk_version() in:
  • Arduino IDE w/ latest ESP8266 download = 2.2.2-dev(38a443e)
  • VSCode/PlatformIO using the "arduino" framework, it gives = 2.2.2-dev(38a443e)
  • VSCode/PlatformIO using the "esp8266-nonos-sdk" framework, it gives = 2.1.0(7106d38)

Brief version - Can you tell me what environment you're using? If you're using a Linux implementation, can you recommend a link for a tutorial to install in Windows? If VSCode/PlatformIO, I can't seem to upgrade my environment to 3.0.5 (current version of NonOS SDK). Most Internet searches return people saying "Why don't you use RTOS". I don't want to use RTOS or the Arduino flavored version.

Detailed Version - I've always used the Arduino IDE version, but I'm getting to the point that it's getting in the way. Mainly I keep running into what appears to be a bug. So, I'm trying to use ONLY native ESP8266 NonOS and even that fails under the Arduino IDE. Switching to NonOS under VSCode fixed the bug. But now I'm running into new issues (this thread) where I want more fine control of the partitions.

The second part of the background - I am offering a Continuing Education class at a local community college for Introduction to IoT. It is a very entry level class, where people are not expected to have any programming experience. Just installing an IDE is a challenge and Linux isn't even in their vocabulary. The Arduino IDE is the go-to so far for its simplicity. VSCode/PlatformIO might be a possible candidate since it at least keeps them on Windows. Asking them to use a Linux VM is pretty much a deal breaker.
 Reply
Return to Native SDK