README_BuildProcess.md · scriptiot/Espruino

Espruino Internals: Build Process

Directories and Files

ChangeLog: What's new
boards/: Information on boards, used to auto-generate a lot of the code
doxygen/: Directory for auto-generated code documentation - see doxygen/README.md for more info
gen/: Auto-Generated Source Files
libs/: Optional libraries to include in Espruino (Math, Filesystem, Graphics, etc)
make/: Makefile fragments for each architecture
misc/: Other useful things
scripts/: Scripts for generating files in gen, and for analysing code/compilation/etc
src/: Main source code
targetlibs/: Libraries for targeted architectures
targets/: Specific code for targeted architectures
tests/: JavaScript Testcases
benchmark/: JavaScript Benchmarks
dist_*: files to be copied into distribution zip file
Makefile: the Makefile which controls the build process

Build Commands

Building for an Espruino board is generally done by typing:

BOARD=BOARDNAME make

And for release, by typing:

RELEASE=1 BOARD=BOARDNAME make

Valid board names are in comments at the head of the Makefile..

Release builds have all the assertions removed, reducing flash usage. Normal builds still have the assertions, and debug builds (with DEBUG=1 have more debug info in the assertions, as well as the debug info needed for proper GDB debugging).

The Make system has the following targets:

clean - Clean the build
all (or unspecified) - Build, including binaries needed for flashing the firmware
flash - Generate the firmware files and flash to the board
serialflash - Generate the firmware files and flash to the board via serial bootloader (used for STM32, especially the Original Espruino and Espruino Pico)

Build Process

Overall build logic is in the Makefile.
The Board Definition File in boards/$BOARD.py is parsed by scripts/get_makefile_decls.py to work out extra definitions for the Makefile (chip family, part number, binary name, whether it has a bootloader), which control the build.
Architecture-specic build logic is in the make directory - this is called based off family in the Board Definition File in boards/.
The Makefile adds source files to $SOURCES, and 'wrapper' source files to $WRAPPERSOURCES. Wrapper files are files that contain functions that are exposed to JS.
The script scripts/build_platform_config.py is run which generates gen/platform_config.h from boards/$BOARD.py - this contains information like the amount of RAM, as well as buffer sizes and the amount of variables that will be stored.
The script scripts/build_pininfo.py creates the pin definitions and puts them in gen/jspininfo.c
The script scripts/build_jswrapper.py is then run on $WRAPPERSOURCES - it generates gen/jswrapper.c - a hard-coded symbol table of built-in functions - from the Wrapper files.
Source files are built
On embedded targets, a linker file gen/linker.ld is auto-generated by scripts/build_linker.py based on the board definition (available flash, ram, bootloader etc). In some targets (especially non-STM32) a pre-made linker file will be used instead.
Everything is linked using link time optimisation (where possible) - this helps to inline code that wouldn't otherwise have been inlined, and generally makes for a much more efficient binary.
The elf file is converted with objdump to:
- bin (for uploading to Espruino board)
- hex (for uploading to mbed/etc)
- lst (assembler listing - used for debugging)
scripts/check_size.sh does a sanity check of the bin file's size against what's described in the board definition, and fails if it won't fit into available flash memory.

Oddities

malloc and free are not included, because they're not needed and by default they use a large pool of valuable RAM to speed up memory allocation.
strcpy, memcpy, memset are all reimplemented in jsutils.h - again because of initial issues with high flash memory usage
Espruino on STM32 uses its own Maths library for double arithmetic, because when building initially the libraries included by GCC dynamically allocated memory - which pulled in malloc. If we could move back to GCC libraries at some point it'd be a huge bonus

Board Definition File

The board definition files are .py files that reside in the boards folder (for example the Pico one). They're used all over the place:

To work out the binary's name
To choose which libraries get built in via the Makefile
To specify the architecture-specific Makefile fragments are used
To create gen/platform_config.h via scripts/build_playform_config.py
To generate the linker file gen/linker.ld using scripts/build_linker.py. Mainly STM32-based targets only.
To create the pin definitions
To create the HTML page on each individual board (referenced from the top of the Reference) using scripts/build_board_docs.py
Converted to JSON (along with the pin declarations) and put on espruino.com to help with autocomplete in the Web IDE (and auto-population of dropdowns in the Blockly editor). Converted by scripts/build_board_json.py

To create a custom board based on one of the distributed board file copy e.g. PICO_P1_3.py to PICO_P1_3_CUSTOM.py and change the content of PICO_P1_3_CUSTOM.py as required for the planed project. The new board file name is than PICO_P1_3_CUSTOM. All created boards with the suffix _CUSTOM.py are ignored by git.

These contain:

info

Board name and link (for the HTML file)
boardname - override board name visible at runtime in process.env.BOARD, otherwise the filename of py file is used
Default console device, pins, and baus rate (the device Espruino goes to when USB is unplugged or not built in)
variables - The number of variables to use (this depends on the amount of RAM available). Less than 1023 vars use 12 bytes per var, more uses 16 bytes. You have to adjust this such that there is spare room for the stack and static variables (on Espruino boards this means leaving around 16kB free, but on smaller boards it can be reduced a lot)
bootloader - whether the binary image needs compiling with a special USB-VCP bootloader (Espruino boards only)
binary_name - the name of the binary that'll be produced
binaries - available binaries - this is used by the Web IDE to allow the user to choose which binary to upload
build - controls what gets build via the Makefile:
- optimizeflags - flags like -O3 to give to the compiler
- libraries - list of libraries to include - these get transformed into USE_LIBNAME defines
- makefile - list of commands/definitions to execute in the Makefile

info.makefile definitions

This is a partial list of definitions that can be added in a BOARD.py file's info.build.makefile array, eg: 'DEFINES+=-DBLUETOOTH_NAME_PREFIX=\'"Puck.js"\''

SAVE_ON_FLASH - Remove some features (like any ES6 support) to target devices with ~128kB Flash
SAVE_ON_FLASH_MATH - Remove some less-used Maths functions that use a bunch of Flash memory
SAVE_ON_FLASH_EXTREME - Pull out as many features as possible to target devices with ~128kB Flash that also want things like Filesystem support
BLUETOOTH_NAME_PREFIX="..." - Make the Bluetooth LE device's name BLUETOOTH_NAME_PREFIX followed by the last 2 bytes of the MAC address.
BLUETOOTH_ADVERTISING_INTERVAL=375 - set the default Bluetooth advertising interval (default 375)
NFC_DEFAULT_URL="http://foo" - If defined, set the advertised NFC URL to the one given, plus ?a=ble_address. Only do it for a fresh boot - not when code has been saved.
PIN_NAMES_DIRECT=1 - Package skips out some pins (maybe there's D0,D1,D3 but no D2), so the code must search rather than just offsetting based on pin number.
DUMP_IGNORE_VARIABLES="...\0" - string containing zero-terminated list of global variable names to ignore when dump() is called. Must be explicityly zero-terminated so there are 2 trailing 0s
FSMC_BITBANG - if using a built-in FSMC Graphics LCD, don't use the hardware but instead do it in software
FLASH_64BITS_ALIGNMENT=1 - For testing 64 bit flash writes on linux
JSMODULESOURCES+=libs/.../foo.min.js - include the given JS file as a module that can be used via require("foo")
JSVAR_MALLOC - Allocate space for variables at jsvInit time, rather than statically
NO_VECTOR_FONT=1 - don't compile in the vector font (this is usually only done for SAVE_ON_FLASH)
NO_DUMP_HARDWARE_INITIALISATION=1 - don't create lines like digitalWrite(D1,1) for dump()/save() to recreate hardware state
USE_FONT_6X8=1 - Also include in a 6x8 fixed width bitmap font
USE_TAB_COMPLETE=0 - Don't include tab completion (default is yes unless SAVE_ON_FLASH is defined)
USE_DEBUGGER=0 - Don't include the debugger (default is yes unless SAVE_ON_FLASH is defined)
USE_NETWORK_JS=0 - Don't include JS networking lib used for handling AT commands (default is yes if networking is enabled)
ESPR_DCDC_ENABLE - On NRF52 use the built-in DCDC converter (requires external hardware)
ESPR_LSE_ENABLE - On NRF52 use an external 32kHz Low Speed External crystal on D0/D1

chip

part - Chip part number (this is defined in the compiler - eg if the part is STM32F401CDU6, -DSTM32F401CDU6 is put on the GCC command-line)
family - Chip family - also defined, but used in the Makefile to define what gets built
package - Used for ST chips when working out Pin Definitions to ensure that the right pins are included
ram on chip in KB
flash on chip in KB
speed of chip in MHz
Number of usarts
Number of spis
Number of i2cs
Number of adcs
Number of dacs
saved_code - how and where to save JS code in the chip's flash memory
- The address of the start of the code in flash memory. Note that on STM32 chips this is at 0x08000000 even though code executes from 0x00000000 (the two areas are mirrored but 0x00000000 is faster to execute from while 0x08000000 is what's needed for writing to flash)
- The page_size in bytes of the first flash page containing code (not used now?)
- The number of flash pages we're using for code (not used now?)
- flash_available in KB, used as a sanity check by scripts/check_size.sh after the build completes
place_text_section what address to start the program code at - This is 0 if not specified, but is useful in devices like the Pico where varying page sizes mean that the program code may be located after the saved program code (and/or bootloader)

board

Names of rows of pins - used solely by scripts/build_board_docs.py to generate the nice HTML description.

There can also be a _css element which contains raw Cascading Style Sheets, which contain carefully tweaked positions of the rows of pins defined in board, to align with the background image in boards/img

devices

This is a list of built-in stuff on the board that is made accessible to Espruino. It's:

Parsed and turned into a series of #defines in gen/platform_config.h by this code.
Used for the magenta tags shown near pins in the board's HTML file - that warn users that the pin might be used for some other function.

Stuff you can use is LED1-LED8, BTN1-BTN4, USB, LCD (for boards with FSMC LCDs built in), SD (SD card), JTAG (when JTAG pins are defined, but we need to make sure we leave them alone when the board resets). You can also define your own.

Pin Definitions

The pin definitions are created by scripts/build_pininfo.py and are stored in gen/jspininfo.c. The script calls get_pins in boards/$BOARD.py and a structure of the following form is returned:

[
   { "name":"PD12", "sortingname":"D12", "port":"D", "num":"20", "functions":{}, "csv":{} },
   { "name":"PD13", "sortingname":"D13", "port":"D", "num":"23", "functions":{ "SPI1_SCK":0 }, "csv":{} },
   ...
]

get_pins can define the raw array (like boards/MICROBIT.py), or on boards with STM32 chips like the Pico some utility functions auto-generate the code from CSV files that were copied from ST's datasheets.

name is the pin name - due to random historical reasons (from ST datasheets) it needs prefixing with P
sortingname is the name, but padded so that when it's sorted everything appears in the right order
port is the actual port - on ESP8266 this might not be needed and could just default to D
num is the pin number - this doesn't have to match D - it's what is needed internally to access the hardware. For instance Olimexino has 'logical' pins that actually map all over the place.
function is a map of pin functions to their 'alternate functions' (an STM32 chip thing - STM32F4 chips can have different peripherals on each pin, so the alternate function is a number that you shove in that pin's register in order to connect it to that peripheral). The format, for instance I2C1_SDA is important as it's parsed later and is used to build gen/jspininfo.c. The code to parse them is here
csv isn't needed, but when using data grabbed from csv files from ST's datasheets like this it contains the raw data for debugging)

Wrapper Files

Wrapper files are normal C source files that contain functions that are exposed to JavaScript code in the interpreter. They're prefixed jswrap_.

See libs/README.md for a short tutorial on how to add your wrapper files.

Each function to be exposed has a specially formatted comment above it containing JSON. The format of this JSON is specified in this file

An example of a wrapper file might be:

#include "jswrap_hello.h"  // We need the declaration of the jswrap_hello_world function
#include "jsinteractive.h" // Pull in the jsiConsolePrint function

// Let's define the JavaScript class that will contain our `world()` method. We'll call it `Hello`
/*JSON{
  "type" : "class",
  "class" : "Hello"
}
Some info about this class
*/

// Now, we define the `jswrap_hello_world` to be a `staticmethod` on the `Hello` class
/*JSON{
  "type" : "staticmethod",
  "class" : "Hello",
  "name" : "world",
  "generate" : "jswrap_hello_world"
}
We can write simple markdown-formatted comments in here. These
then get scanned and turned into the Espruino [Reference](http://www.espruino.com/Reference)

 * Bulletted
 * List
*/
void jswrap_hello_world() {
    jsiConsolePrint("Hello World!\r\n");
}

In the build process, scripts/build_jswrapper.py parses all these files, pulls out the headers, and generates gen/jswrapper.c - a hard-coded symbol table that resides on flash memory. It's consulted whenever

scripts/build_jswrapper.py also needs to be aware of what definitions were passed to the compiler - for instance if SAVE_ON_FLASH is defined, several non-vital functions will not be compiled in (so should not appear in the symbol table).

The wrapper files are also parsed by:

scripts/build_docs.py, which builds the HTML file used for the Espruino Reference.
scripts/build_tern_json.js, which generates a JSON description of all functions (along with their documentation) which is used for code completion in the Web IDE and to parse all code examples in order to produce the 'Examples' links in the Reference.

scriptiot / Espruino

Espruino Internals: Build Process

Directories and Files

Build Commands

Build Process

Oddities

Board Definition File

info

info.makefile definitions

chip

board

devices

Pin Definitions

Wrapper Files

简介

发行版

贡献者

近期动态

scriptiot / Espruino .gitee-modal { width: 500px !important; }

Espruino Internals: Build Process

Directories and Files

Build Commands

Build Process

Oddities

Board Definition File

info

info.makefile definitions

chip

board

devices

Pin Definitions

Wrapper Files

简介

发行版

贡献者

近期动态

搜索帮助

scriptiot / Espruino