Setting up the ESP32 and ESP8266 build environment

07 Oct 2019 - tsp
Last update 28 Nov 2019
Reading time 5 mins

TL;DR: There is a shellscript that automates everything except the machine setup at the bottom (but as mentioned this only works on Linux - and requires a single sudo apt-get install gcc git wget make libncurses-dev flex bison gperf python python-serial before)

The following short summary shows how on can easily set up an development environment for the ESP8266, ESP8285 and their larger relative the ESP32. The ESP8266 and ESP8285 are built around the 80 MHz clocked Tensilica Xtensa LX3 core (ESP8285 has internal flash memory in constrast to ESP8266 which just used external flash) and support WiFi connectivity, the ESP32 is built around a dual core 240 MHz Tensilica Xtensa LX6 that additionaly supports Bluetooth and with external PHY also Ethernet connectivity. Both are programmed using a derivative of the GNU Compiler Collection. They are also supported by the Arduino IDE. One can simply get started with one’s own design or - if one’s less interested in the hardware - with one of the following development boards (note that all links are Amazon affilate links - this pages author profits from qualified purchases):

The ESP8266 NodeMCU Amica
The ESP32 NodeMCU module
The WiFi Kif 8 development module including an ESP8266 and an OLED display

First let’s face a sad truth: The environments are built to run only on Linux, not to be portable accross various development systems (especially not across other Unix systems which is somewhat sad in my opinion). This is caused by assumptions inside shellscripts and prebuilt binaries that are spread over the development environments.

So first one has to setup a Linux machine or VM somewhere. I’ve decided to use a decent Ubuntu image excluding the graphical user interface. For me moving to a Linux machine just feels like somewhat Unix like but way more chaotic than other systems like FreeBSD - but that’s a requirement when using Espressif’s and Xtensas toolchain.

After that one has to install dependencies:

sudo apt-get install gcc git wget make libncurses-dev flex bison gperf python python-serial

Then one downloads and installs the Xtensa toolchain (that contains binary prebuilts):

mkdir -p ~/esp
cd ~/esp
wget https://dl.espressif.com/dl/xtensa-lx106-elf-linux64-1.22.0-100-ge567ec7-5.2.0.tar.gz
tar -xzf xtensa-lx106-elf-linux64-1.22.0-100-ge567ec7-5.2.0.tar.gz

After that the required SDKs. The following three commands fetch (in that order)

The RTOS SDK for ESP8266 to develop for ESP8266 using
The NO-OS SDK for ESP8266 to develop on +- bare metal
IDF for ESP32 development

git clone --recursive https://github.com/espressif/ESP8266_RTOS_SDK.git
git clone --recursive https://github.com/espressif/ESP8266_NONOS_SDK
git clone --recursive https://github.com/espressif/esp-idf.git

Then one has to build the esp-idf if one wants to develop for ESP32 - which is done by using their install scripts.

cd esp-idf
./install.sh
cd ..

For ESP8266 RTOS SDK one has to install the required Python modules:

python -m pip install --user -r ESP8266_RTOS_SDK/requirements.txt

After that only environment variables have to be set as usual. The PATH variable has to be extended by the Xtensa toolchain for every target. This can be done in .profile as usual.

export PATH="${PATH}:`pwd`/xtensa-lx106-elf/bin

To add the path in your profile you can add the following to ~/.profile using an editor like nano or vi:

if [ -d "${HOME}/esp/xtensa-1x106-elf/bin"] ; then
	PATH="${PATH}:${HOME}/esp/xtensa-1x106-elf/bin"
fi

Note that each development environment uses the IDF_PATH variable so the choice depends on which target you are using.

ESP8266 RTOS SDK: export IDF_PATH=`pwd`/ESP8266_RTOS_SDK/
ESP8266 NON-OS SDK: export IDF_PATH=`pwd`/ESP8266_NONOS_SDK/
ESP32 IDF: ./esp-idf/export.sh

Automating everything

Since we use an automated build and deployment system we’ve written a short shellscript that automates fetching and preparing build environments for the different targets (this is nearly the same script we use except that there is a script running before that creates the buildbots users, registers SSH keys and installs the dependencies with apt-get as described above).

#!/bin/sh
set -e

if [ $# -lt 1 ]; then
        echo "\e[31mPlease specify target (ESP8266NONOS, ESP8266RTOS, ESP32IDF)\e[39m"
        return 1
fi

export TARGET="0"
if [ "$1" = "ESP8266NONOS" ]; then
        export TARGET="NONOS8266"
fi
if [ "$1" = "ESP8266RTOS" ]; then
        export TARGET="RTOS8266"
fi
if [ "$1" = "ESP32IDF" ]; then
        export TARGET="RTOS32"
fi

if [ "${TARGET}" = "0" ]; then
        echo "\e[31mUnknown target ${1}. Plase specify one of ESP8266NONOS, ESP8266RTOS, ESP32IDF\e[39m"
        return 1
fi

olddir=`pwd`

if [ ! -d ${HOME}/esp ]; then
        mkdir -p ${HOME}/esp
fi


# Check toolchain is already present
echo "\e[33mChecking toolchain presence ...\e[39m"

if [ ! -d "${HOME}/esp/xtensa-lx106-elf" ]; then
        echo "\e[33mFetching toolchain\e[39m"

        cd ${HOME}/esp
        wget https://dl.espressif.com/dl/xtensa-lx106-elf-linux64-1.22.0-100-ge567ec7-5.2.0.tar.gz
        tar -xzf xtensa-lx106-elf-linux64-1.22.0-100-ge567ec7-5.2.0.tar.gz

        export PATH="${PATH}:${HOME}/esp/xtensa-lx106-elf/bin"
fi

echo "\e[32mToolchain is available\e[39m"

# Fetch SDKs from github
echo "\e[33mFetching SDKs from GitHub if required\e[39m"
if [ ! -d "${HOME}/esp/ESP8266_RTOS_SDK" ]; then
        cd ${HOME}/esp
        git clone --recursive https://github.com/espressif/ESP8266_RTOS_SDK.git
        echo "\e[32mFetched ESP8266 RTOS SDK\e[39m"
fi

if [ ! -d "${HOME}/esp/ESP8266_NONOS_SDK" ]; then
        cd ${HOME}/esp
        git clone --recursive https://github.com/espressif/ESP8266_NONOS_SDK
        echo "\e[32mFetched ESP8266 NON-OS SDK\e[39m"
fi

if [ ! -d "${HOME}/esp/esp-idf" ]; then
        cd ${HOME}/esp
        git clone --recursive https://github.com/espressif/esp-idf.git
        echo "\e[32mFetched ESP-IDF for ESP32\e[39m"
        cd esp-idf
        ./install.sh
        echo "\e[32mInstalled ESP-IDF for ESP32\e[39m"
fi

echo "\e[32mAll SDKs available\e[39m"
echo "\e[33mInstalling required python modules\e[39m"

cd ${HOME}/esp
python -m pip install --user -r ${HOME}/esp/ESP8266_RTOS_SDK/requirements.txt

echo "\e[32mDone, now exporting IDF path for selected target\e[39m"

if [ "${TARGET}" = "NONOS8266" ]; then
        export IDF_PATH=${HOME}/esp/ESP8266_NONOS_SDK
        echo "export PATH=${PATH}:${HOME}/esp/xtensa-lx106-elf/bin"
        export PATH=${PATH}:${HOME}/esp/xtensa-lx106-elf/bin
fi
if [ "${TARGET}" = "RTOS8266" ]; then
        export IDF_PATH=${HOME}/esp/ESP8266_RTOS_SDK
        echo "export PATH=${PATH}:${HOME}/esp/xtensa-lx106-elf/bin"
        export PATH=${PATH}:${HOME}/esp/xtensa-lx106-elf/bin
fi
if [ "${TARGET}" = "RTOS32" ]; then
        export IDF_PATH=${HOME}/esp/esp-idf
        export PATH=${PATH}:${HOME}/.espressif/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/
        echo "export PATH=${PATH}:${HOME}/.espressif/tools/xtensa-esp32-elf/esp32-2019r1-8.2.0/xtensa-esp32-elf/bin/"
fi

echo "\e[32mDone\e[39m"
echo "export IDF_PATH=${IDF_PATH}"

if [ $# -gt 1 ]; then
        # Execute passed command
        SUBCOMMAND=${2}
        shift
        shift
        echo "\e[32mRunning ${SUBCOMMAND} $@\e[39m"
        if [ $# -gt 0 ]; then
                env PATH=${PATH} IDF_PATH=${IDF_PATH} ${SUBCOMMAND} $@
        else
                env PATH=${PATH} IDF_PATH=${IDF_PATH} ${SUBCOMMAND}
        fi
fi