-->
Page 1 of 1

Tuning the build - useful utility

PostPosted: Thu Mar 21, 2019 7:36 pm
by davydnorris
All,

Just thought I would pass this on. I was looking for something to help analyse the map files generated by the gcc linker and found this. It's been really very useful

https://www.embeddedrelated.com/showarticle/900.php

Basically fire it up, change the settings to point to the ESP compiler's nm and readelf, and then point to the map file and the .out file that's created before the esptools turn it into a flash image.

The one thing I would love is to be able to add sections to the list - right now I have to adjust the settings to see the irom

Re: Tuning the build - useful utility

PostPosted: Mon Mar 25, 2019 2:00 pm
by eriksl
And I was just getting about excited ;-)

I have a small perl script in my makefile which does this, it lists all symbols in e.g. IRAM and sorts them by size. But it doesn't work anymore, since all libs are now compiled with -ffunctions-sections and -fdata-sections and also the linker map appears to use continuation lines, which my script doesn't handle really.

So I guess that still leaves some work for me.

I'll share the script when it's finished. I think it's very important to know what is exactly filling you IRAM.

Re: Tuning the build - useful utility

PostPosted: Mon Mar 25, 2019 4:46 pm
by davydnorris
I'm going to tweak this one up a bit, or use it to build my own.

I've also found another one which is better in some way but not as good in others:

http://www.sikorskiy.net/prj/amap/

Once I'm out of this release there's a list of things I want to do!!!

Re: Tuning the build - useful utility

PostPosted: Mon Apr 01, 2019 1:53 pm
by eriksl
In the meantime I crafted up this perl script. It does not use the linkmap. I found the linkmap to be very hard to parse reliably in a script. So what I do now is to run "nm" (can be your own nm, no need for the lx106 one) on the final *.o file (the one that goes into esptool(2) for image generation). And then collect all symbols them, sort them by section (not region!) and order them by size.

One remark though: the size of individual symbols if notoriously hard to establish. This script collects the size as stated from "nm" and also calculates the size, as difference from the symbol's address and the next one. Both of them are not 100% reliable. The first does not include alignment/padding and is sometimes wrong for no apparent reason. The second one does include the padding, but is also wrong sometimes, probably because there are symbols somehow "hidden" for nm.

Having said that, it has helped me greatly to clean up both DRAM (move rodata to flash) and IRAM (move functions to flash). Free DRAM went from 4k to 12k, even though almost all strings were already in flash. I'd say, give it a go...

Code: Select all#!/usr/bin/perl -w

no warnings 'portable';

use Data::Dumper;

my($input) = $ARGV[0];
my($fd);
my($symbol, $address, $class, $type, $size, $line, $section, $source);
my(%symbols, %value);

my(%sections) = (
   "bss" =>
   {
      "data" => 1,
      "start" => hex("3ffe8000"),
      "end" => hex("3fffbfff"),
   },
   "data" =>
   {
      "data" => 1,
      "start" => hex("3ffe8000"),
      "end" => hex("3fffbfff"),
   },
   "irom0.text" =>
   {
      "data" => 0,
      "start" => hex("0x40200000"),
      "end" => hex("0x402fa000"),
   },
   "rodata" =>
   {
      "data" => 1,
      "start" => hex("0x3ffe8000"),
      "end" => hex("0x3fffbfff"),
   },
   "text" =>
   {
      "data" => 0,
      "start" => hex("0x40100000"),
      "end" => hex("0x40108000"),
   },
);

my(%class_to_text) = (
   "A" => "global absolute",
   "a" => "local absolute",
   "B" => "global bss",
   "b" => "local bss",
   "C" => "common",
   "D" => "global data",
   "d" => "local data",
   "G" => "global sdata",
   "g" => "local sdata",
   "I" => "indirect",
   "N" => "debug",
   "n" => "comment",
   "p" => "stack unwind",
   "R" => "global rodata",
   "r" => "local rodata",
   "S" => "global sbbs",
   "s" => "local sbbs",
   "T" => "global text",
   "t" => "local text",
   "U" => "undefined",
   "u" => "unique",
   "V" => "WEAK1",
   "v" => "weak1",
   "W" => "WEAK2",
   "w" => "weak2",
   "-" => "stab",
   "?" => "unknown",
   "" => "",
);

die("failed to start nm") if(!open($fd, "nm -n -l -a -fsysv --synthetic " . $input . " |"));

while(<$fd>)
{
   chomp();

   ($symbol, $address, $class, $type, $size, $line, $section, $source) =
      m/^([^\s|]+)\s*\|\s*([0-9a-f]+)\s*\|\s*([^\s]+)\s*\|\s*([^\s|]*)\s*\|\s*([0-9a-f]*)\s*\|\s*([\^s|]*)\s*\|\s*([a-zA-Z0-9._*]*)\s*(.*)$/o;

   next if(!defined($symbol) || !defined($address) || !defined($class) || !defined($type) || !defined($size) || !defined($line) || !defined($section) || !defined($source));
   next if($section !~ m/^\.[a-zA-Z]+/g);

   $section =~ s/^.//o;

   undef(%value);
   $value{name} = $symbol;
   $value{address} = hex($address);
   $value{size_1} = hex($size);
   $value{class} = $class;
   $value{type} = $type;
   ($value{source_file}, $value{source_line}) = $source =~ m/\/([^\/]+):([0-9]+)$/o;

   %{$symbols{section}{$section}{by_name}{$symbol}} = %value;
   %{$symbols{section}{$section}{by_address}{hex($address)}} = %value;
}

close($fd);

my($sections_entry, $previous_address, $this_address, $previous, $this, $key);

for $section (sort(keys(%{$symbols{section}})))
{
   undef($previous_address);

   if(!exists($sections{$section}))
   {
      printf("section %s is unknown\n", $section);
      next;
   }

   $sections_entry = \%{$sections{$section}};
   $$sections_entry{size_1} = 0;
   $$sections_entry{size_2} = 0;

   for $this_address (sort(keys(%{$symbols{section}{$section}{by_address}})))
   {
      if(!defined($previous_address))
      {
         $previous_address = $this_address;
         next;
      }

      $previous =   \%{$symbols{section}{$section}{by_address}{$previous_address}};
      $this =      \%{$symbols{section}{$section}{by_address}{$this_address}};

      if(($$this{address} < $$sections_entry{start}) || ($$this{address} > $$sections_entry{end}))
      {
         printf("   invalid address for symbol %s for section %s: %x\n", $$this{name}, $section, $$this{address});
         next;
      }

      $$previous{size_2} = $$this{address} - $$previous{address};
      $$sections_entry{size_1} += $$previous{size_1};
      $$sections_entry{size_2} += $$previous{size_2};
      $key = sprintf("%-10s:%4x:%s", $section, $$previous{size_2}, $$previous{address});
      %{$symbols{section}{$section}{by_size}{$key}} = %{$previous};

      $previous_address = $this_address;
   }

   $$this{size_2} = $$this{size_1};
   $$sections_entry{size_1} += $$this{size_1};
   $$sections_entry{size_2} += $$this{size_1};

   $key = sprintf("%-10s:%4x:%s", $section, $$this{size_2}, $$this{address});
   %{$symbols{section}{$section}{by_size}{$key}} = %{$this};
}

my(%section_to_region) =
(
   "bss" => "dram",
   "data" => "dram",
   "rodata" => "dram",
   "text" => "iram",
   "irom0.text" => "flash",
);

my($region, %region);

printf("sections\n");

for $section (qw[bss data rodata text irom0.text])
{
   $region = $section_to_region{$section};

   printf(" %-10s used: %6d region: %s\n",
         $section,
         $sections{$section}{size_2},
         $region);

   $region{$region}{name} = $region;
   $region{$region}{start} = $sections{$section}{start};
   $region{$region}{end} = $sections{$section}{end};
   $region{$region}{size} += $sections{$section}{size_2};
}

printf("\nregions\n");

my($length);

for $region (qw[dram iram flash])
{
   $length = $region{$region}{end} - $region{$region}{start};
   $free = $length - $region{$region}{size};

   printf(" %-10s start: 0x%8x end: 0x%8x length: %7d used: %6d free: %6d %3d%%\n",
         $region,
         $region{$region}{start},
         $region{$region}{end},
         $length,
         $region{$region}{size},
         $free,
         ($free * 100) / $length,
      );
}

printf("\nsymbols\n");

my($name, $section2, $source_line, $source_file);

printf(" %-10s %-6s %-14s %-8s %-5s %-5s %-44s %-4s %s\n", "section", "region", "class", "address", "size1", "size2", "name", "line", "file");

for $section (qw[bss data rodata text irom0.text])
{
   $region = $section_to_region{$section};

   for $key (sort(keys(%{$symbols{section}{$section}{by_size}})))
   {
      ($section2, $size, $address) = $key =~ m/^([^\s]+)\s*:\s*([0-9a-f]+)\s*:(.*)/o;

      $this = \%{$symbols{section}{$section}{by_address}{$address}};

      $source_line = $$this{source_line};
      $source_file = $$this{source_file};

      $source_file = "" if(!defined($source_file));

      if(!defined($source_line))
      {
         $source_line = "";
      }
      else
      {
         $source_line = sprintf("%4s", $source_line);
      }

      printf(" %-10s %-6s %-14s %08x %5d %5d %-44s %-4s %s\n", $section2, $region, $class_to_text{$$this{class}}, $address, $$this{size_1}, $$this{size_2},
            $$this{name}, $source_line, $source_file);
   }
}