How to set up manually the GCC toolchain and SDK

An opensource toolchain is available in https://github.com/pfalcon/esp-open-sdk. It is ready to run on Linux hosts only. If you don't use a Linux development machine, then the easiest way to build and to use the toolchain is to set up a Linux Virtual Machine (VM) on your development machine, as described in the following section. Another alternative is to use a low cost Linux board such as a Raspberry Pi 2. Once you have your working Linux environment, setting up the toolchain is easy.

• Use Git command to clone the git repository esp-open-sdk into your local directory:

  git clone --recursive https://github.com/pfalcon/esp-open-sdk

• Review the open SDK's README.md and edit line 3 of the Makefile to select which the Espressif SDK Version that you wish to install if it is not the (default) latest version, e.g. VENDOR_SDK = 1.3.0

• Run the make to install. This will take a long time, especially on a slow internet connection, so take a coffee . Note that we recommend a standalone variant install as Espressif update their SDK quite frequently, keeping the log just in case you want to check it.

  make STANDALONE=y |& tee make0.log

• You will need to include the xtensa-lx106-elf/bin in your PATH to use the tools. The simplest way of doing this is to include the following in the .bashrc in your login directory.

  export PATH="$HOME/esp-open-sdk/xtensa-lx106-elf/bin/:$PATH"

• Finally log in again and run xtensa-lx106-elf-gcc -v to validate the install. Note that if you type in toolchain commands at the command prompt, then you might wish to add frequent aliases for the toolchain gcc, ar, objdump etc. in your .bashrc to save on typing, for example:

  alias xgcc="xtensa-lx106-elf-gcc"

If you run Windows, OSX or Linux 64bit, then one of the easiest ways of setting up a development is to use a virtualization product to run a dedicated toolchain VM. There are a number of products available, but my personal preference is Oracle's VirtualBox as this is free, flexible and performs well on most common Intel / AMD platforms. This brief overview is written assuming the use of VirtualBox, but largely applies to setting up VM on most platforms.

• Since nearly all toolchain work is done at the command prompt, it is simpler to configure your VM as a server and access it over SSH using the ssh command on Linux or OSX and a product such as puTTY on Windows. If you decided to go this route then you don't need to learn the a Linux windows environment since you can access and manipulate both files and console sessions through your familiar Windows environment.
• A simple VM with 384Mb RAM and 8Gb disk is easily sufficient to develop and build EESP8266 SDK-based images, and to keep thing simple I suggest using minimal simple 64bit Ubuntu server configuration which mirrors the one that the nodemcu project uses to test its builds.
• The first thing that you need to do is to download the VM product. In the case of VirtualBox, you don't need any of Oracle's closed source layered extension packs, so you can do a sudo apt-get install virtualbox on Debian and derived Linux systems such as Ubuntu. In the case of Windows and OSX, follow the instructions on the product's Downloads page or follow one of the many step-by-step guides on Youtube and elsewhere.
• You now need to download the Ubuntu Minimal installation CD for Ubuntu 12.04 LTS Amd64 mini.iso. (This is less than 30Mb). NodeMCU uses the Travis-CI system to validate the build integrity of any changes to its Github repostiory; Travis also uses Ubuntu 12.04 VMs, so NodeMNCU contains a pre-built toolchain for this VM configuration in its tools folder and unpacking this toolchain is a lot simpler and takes up about 2½Gb less disk space than building it yourself.
• VirtualBox has both a complete GUI and commandline interface, but again the simplest thing to do is to use the VirtualBox GUI and follow one of the tutorials to create a new virtual machine (which I will call nodemcu, but you are free to choose whatever name your wish) (the bold options are non-default):

  • 64-bit Ubuntu; 384Mb RAM.
  • 1 CPU; PAE disabled; and enable H/W virtualization assist if available.
  • Set a minimal video memory (4Mb and no fancy features).
  • Enable a IDE CD/DVD and map the mini.iso to this.
  • Enable and create a VDI-based dynamically allocated 8Gb SATA HDD. (You might want to place the VDI file in a specific work folder rather leaving it hidden in the VirtualBox folder hierarchy.)
  • You will need one NAT-based network adapter.
  • You don't need any other virtual hardware or features, many some find the shared folders feature worth installing. I don't bother nowadays and just use rsync / scp / sftp.
• NAT is described in the VirtualBox documentation, but in essence the VM host application acts as a router, so the VM doesn't have its own IP address. Instead you map host IP ports to client posts through the Network → Advanced → Port Forwarding function and a common port is to forward 127.0.0.1:2222 to 10.0.2.15:22 so that doing an SSH to localhost:2222 connects to the VM's default SSH port. Host and off-machine process can only access the VM through a mapped port when the VM is running. (You will need to do an ifconfig after you've built the machine to validate the guest's actual IP address.) Note that if you have multiple VMs, then you should use a unique port for each SSH eg. 2222 for VM1; 2322 for VM2, etc. This stops you shooting yourself in the foot.
  
• You now start VM through the GUI and you will boot into the Ubuntu minimal install. There are again various YouTube and other tutorials which will take you through this, but what you need to do is to build a minimal server system that you can log onto over SSH.
  • Use the arrow / tab /enter keys to select Install.
  • Choose your language.
  • Chose your keyboard. Don't use automatic; just do a manual selection.
  • Detect network and disable USB storage as you won't need it.
  • Give it the same host name as your VM machine name to keep things simple.
  • Choose a local Ubuntu mirror, and step through the menus until you get to Download Installer Components and continue as you don't need any of these. The installer occasionally goes silent for 30-60secs, but then downloads all of the core components over network from the mirror. How long this takes depends on your internet bandwidth and how heavily the mirror is being hit, but 10-15 mins is typical. Time for a coffee
  • Now let the menu step you through the inputs. The defaults nearly always apply. You need to choose a user name and password. Keep the name the same as you use on your host. Don't bother encrypting your home directory and choose a simple password (as you will only use it 5 mins). Use NTP to set the time.
  • Keep the disk partitioning simple: select guided / use whole disk / All files in One Partition and install the base system. Another coffee or whatever, as this will chug along for another 20 mins or so. Sorry guys, but this is a LOT less than Windows 10
  • When it comes to configuring the package manager, I untick everything, and then I select no automatic updates, for reasons given below,
  • Only check Basic Ubuntu Server and openSSH Server on the Software Selection menu and hit continue to finish the install. You are then prompted to remove the media which you do virtually through a VirtualBox menu option, and the installer then reboots the machine.
  • At this point you can sudo ipconfig to check the IP details and sudo poweroff to shutdown the machine.
  • The basic server build is now complete.
• I suggest that you run the server from now on in what is called headless mode – that is you start and stop the VM from a command line using vboxmanage commands and just like a real server, you will only need to have the server console available for exceptional system maintenance. (I never bother with it and do everything over SSH.) I use a set of aliases in my .bashrc on my laptop. (Windows users can use doskey macros to achieve the same effect.) (Replace nodemcu by whatever you've called your VM.

    alias nodemcustart='vboxmanage startvm nodemcu --type headless'
    alias nodemcusave='vboxmanage controlvm nodemcu savestate'
    alias nodemcu='ssh -p 2222 localhost'

• A major advantage of the network install is that the latest versions of all packages at the point of the build are loaded into the system, so there's no need to do the 100s of patch updates that you typically need to do with Windows. We are going to lock the system down so that the only access to the system will be from your account on the host PC or laptop; in effect, no ports are exposed to the wider network. The only way that the VM can be security compromised is if the host PC and host account have been compromised so there is no point in adding extra tiers of security within the VM.
• Start the server and log on using your username / password combination. (If using SSH or puTTY for the first time google “putty using ssh keys authorized_keys” and read up on how public key based SSH access works and is configured). Now copy your public key to ~/.ssh/authorized_keys and chmod 600 ~/.ssh/authorized_keys. You should now be able to log off and log on from your host without needing to enter a password.
• Edit /etc/ssh/sshd_config and append the line AllowUsers *@10.0.2.* to limit logon to access from the host PC. (You will neeed to use sudo nano or sudo vi to do this). And similarly do a sudo visudo to add NOPASSWD to the %admin and %sudo groups as follows; this enables you to access root privileges without having to enter a password (and log off and on again to check that these are working OK.)

    %admin ALL=(ALL) NOPASSWD: ALL
    %sudo  ALL=(ALL:ALL) NOPASSWD: ALL

• And lastly mangle your password so that only key-based access can work:

    pwd=$(date|md5sum); echo -e "$pwd\n$pwd"|sudo passwd $USER

• Now install all of the other core packages that allow you to build the toolchain, nodeMCU and luac.cross. Note that Ubuntu 12.04 doesn't include pre-packaged versions of some of the Lua libraries, so you need to do a luarocks install to build these:

  sudo apt-get install \
     build-essential zip gdb git vim make unrar autoconf automake gawk \
     bison texinfo libtool gcc g++ gperf libc-dbg ncurses-dev expat flex help2man \
     lua5.1 lua5.1-doc luarocks
  for m in bitlib luafilesystem md5 luaposix luasocket; do sudo luarocks install $m; done 

and now you are good to go. If you want to work on the nodeMCU firmware documentation then you will also need the MkDocs toolkit:

  sudo apt-get install python-pip
  sudo pip install mkdocs

See the MkDocs Website for more info.

Another Integrated Development Environment (IDE) for programming in C++ and more easier for beginners is the Arduino IDE http://www.arduino.cc/en/Main/Software. Arduino IDE is an editor that is able to compile the code for us using Xtensa GCC toolchain, which is available at github as ESP8266/Arduino http:https://github.com/esp8266/Arduino. It also uses the ESPTool to automatically upload the hex to the SOC (system on chip). This repository provides several examples that you can put on the chip, and describes the install instructions for running it. They are as simple as adding the board manager url http://arduino.esp8266.com/package_esp8266com_index.json in the preference settings menu and install it using Tools→Board→Board Manager.

An arduino board is not needed as the ESP8266 has 16 times the memory and 5 times the speed.

32KB for the UNO Flash and it operates at 16 Mhz.

512 KB for the ESP8266 and it operates at 80 Mhz. (Up to 4Mb and 160Mhz on some)

Its limitation is its IO pins . Most have but one Analog pin but many Digital pins