First, I've been profiling the memory performance using this technique:
testBaseline.lua:
Code: Select allprint("start", node.heap())
config = flashMod("config")
print("before", node.heap())
wifi.sta.getap(function() print("hi") end)
print("after", node.heap())
testConfig.lua:
Code: Select allprint("start", node.heap())
config = flashMod("config")
print("before", node.heap())
wifi.sta.getap(function(t) config.ListAP(t) end)
print("after", node.heap())
Note that they are identical except for the test version calls the config.ListAP in the callback, and the baseline version calls print("hi"). This gave me a way to determine if it was the function taking up heap or other lua overhead that I don't have control over, and both methods had nearly identical memory utilization, even on the original approach.
However, I did notice I was leaking some memory on the persistent environment of private and public methods, that the 4-way chain of __index calls was slowing things down even more than loading from flash, and a few other issues that the original approach had, so I decided to distill the approach down to its simplest possible form, and go for the lightest implementation I could imagine. And here is what I came up with:
Code: Select allflash = {MOD_NAME = "flash"}
function flash.flashMod(tbl)
if type(tbl) == "string" then tbl = {MOD_NAME=tbl} end
for k,v in pairs(tbl) do
if type(v) == "function" then
file.open(string.format("%s_%s.lc", tbl.MOD_NAME, k), "w+")
file.write(string.dump(v))
file.close()
tbl[k] = nil
end
end
return setmetatable(tbl, {
__index = function(t, k)
return assert(loadfile(string.format("%s_%s.lc",t.MOD_NAME,k)))
end
})
end
flash.flashMod(flash)
flash = nil
module = nil
package = nil
newproxy = nil
require = nil
collectgarbage()
function flashMod(tbl) return loadfile("flash_flashMod.lc")(tbl) end
Notice a few things here - I've completely gotten rid of the concept public/private/flash. Everything is considered a flash function at the time flashMod is called. So, if you want some in-memory functions, add them to the table after you call flashMod... simple enough. Also, it is no longer setting the environment of the flash functions, so there is no concept of a private variable. You could easily pass an environment table as the second argument to flashMod, and use it inside the __index function to set the environment of the function, but this creates a closure around the __index function, bloating memory requirements for a feature that is a nice to have. I'd suggest marking private functions and data with an _, or only accessing them from in-memory functions, which do have access to the local namespace of the module.
Also, flashMod has become a global function which just bootstraps the flashMod function into existence using its own technique. The idea was to replace the module/package/require functionality with a single flash function, so I went ahead and deleted those other functions from the global namespace, freeing up almost another 1k of heap.
The special value MOD_NAME has been added to the table to hold the name of the module (the first half of the flash function file name). This was done to remove another closure, reducing memory requirements of the function that does have to stay in memory: __index. At this point, I could move the metatable into the global environment, and reference the same metatable from every flash module. I'll have to do some profiling to see if this helps once I have several flash modules to work with.
flashMod can be called in two ways:
serializes all the functions on the table to flash, removes them, and augments the table with a metatable that loads the functions on demand.
creates an empty table with the metatable populated. Use this if you want to create the module in one file with the first version, but don't need to attach any additional in-memory methods or data.
So, a full module with public, private, and flash methods will take this form now:
Code: Select allmod = { MOD_NAME = "my_mod"}
function mod.flashFunc1() end
function mod.flashFunc2() end
flashMod(mod) --everything before this is a flash function, everything after it is an in-memory public function
local function privateFunc() end --only visible to public and other private functions
local privateData --ditto
function mod.publicFunc() end
return mod
The zip attached to the first post has the new version of the config module mentioned in previous posts, as well as a flash version of LLbin (I was tired of waiting for a dofile() to load LLbin, so now I have it as a flash function with a global wrapper that is loaded during init. Faster binary transfer startups with hardly any heap usage). Feel free to play with this new version.
Let me know what you think.
David