Monday, January 18, 2016

C.H.I.P. Flashing under Gentoo Linux

Well, it's time to give some credit where credit is due -- the people at Next Thing Co. have been pretty responsive about the boot issues with the C.H.I.P. $9 computer.  They are actively looking into ways to make reflashing the device easier, and have given some updated instructions about how to do the reflash.

In the name of hoping to help someone rather than complain, I would like to give some instructions on how to reflash the C.H.I.P. under Gentoo Linux without the need to install the entire SDK.  This should also give those who are interested in some of the particulars of the flash process a starting point to how it all works.

Step 1:  RTFWP (Read The Fine Web Pages)

Next Thing Company (NTC) has two web pages that you should familiarize yourself with before going through this process.  Don't do anything just yet - just read over how the process works.  It will help to understand why this process is actually a bit easier.

http://docs.getchip.com/#flash-chip-firmware
https://bbs.nextthing.co/t/solution-for-c-h-i-ps-that-do-not-boot/1973

Step 2:  Choose a location to work - make a new subdirectory, and make that a nice clean workspace.  Get into that directory.

Step 3:  Get the sunxi-tools and compile them, as specified in the instructions.  Note that Gentoo has a masked version of sunxi-tools which may work (I didn't bother trying this, but it may work for you).  This was relatively simple:

(from your "working directory")
$ git clone https://github.com/linux-sunxi/sunxi-tools.git sunxi-tools
$ cd sunxi-tools ; make
$ for i in bootinfo fel fexc nand-part pio ; do ln -s sunxi-$i $i ; done


The last line is needed because the utilities are all prefixed by "sunxi-" but the C.H.I.P. flashing tools reference them unprefixed.

The sunxi-tools are utilities that are designed to speak to the Allwinner ARM-based SoC that is used in the C.H.I.P.  The "FEL" mode is, as I understand it, a diagnostic mode of the SoC that allows it to be manipulated in various ways outside of its normal operation (kind of like JTAG, but more powerful and less mysterious).

Step 4:  Get CHIP-tools -- These are the shell scripts that actually orchestrate the firmware flashing.  You'll want to look at these sometime as it provides some insight on the flash process.

(also from your "working directory")
$ git clone https://github.com/NextThingCo/CHIP-tools

You'll note that in the README.md file, the folks at NTC said all you'll need are the sunxi-tools and U-boot tools.  Well, as you'll see in the next step, that isn't entirely correct...

Step 5:  Install the Android tools and an extra thing...  The scripts that do the flashing need to put the binaries in the proper format and talk to the SoC in a way that I'm still not sure about.  Suffice to say that two utilities are needed in this process, and that's fastboot and img2simg.  The fastboot utility seems like a swiss-army knife for talking to Android devices, and apparently the C.H.I.P. as well.  The img2simg utility converts a boot image into a "sparse" image, in 2M chunks.  I'm not sure why this is done this way, but it's needed.

On Gentoo, you'll first want to create/edit a package.keywords file to permit the newer version of android-tools to be built (dev-util/android-tools-5.1.1_p13).  You'll have something like this:

=dev-util/android-tools-5.1.1_p13       ~amd64 ~x86 ~arm

Then emerge android-tools normally.

Unfortunately, the img2simg utility is not compiled in the Gentoo ebuild, so you'll need to do this by hand.  The easiest (not the most efficient) way to do this is:

# cd /usr/portage/dev-util/android-tools
# ebuild android-tools-5.1.1_p13 unpack
# cd /var/tmp/portage/dev-util/android-tools-5.1.1_p13/work/core/libsparse
# cc -I include -lz -o img2simg img2simg.c backed_block.c \
         output_file.c sparse.c sparse_crc32.c sparse_err.c sparse_read.c

You will end up with a nicely compiled img2simg utility, which I moved to the CHIP-tools working directory because it will be in the path in a moment...  Remove the /var/tmp/portage/dev-util/android-tools tree when you're done (it just consumes unnecessary space).

Step 6:  Install u-boot-tools.  These are used to format some of the boot images into the proper format for flashing, since the C.H.I.P. uses U-Boot as its bootloader.  Again, since we'd like to use a newer version of U-Boot tools than Gentoo has agreed is stable, we'll add this to a package.keywords file:

=dev-embedded/u-boot-tools-2015.04     ~amd64 ~x86 ~arm

Then emerge u-boot-tools normally...

Important note:   If you're planning on doing the flashing as an unprivileged (non-root) user, you'll want to add that user to the usb group in /etc/group (and that user will need to do a fresh login for it to take effect) so that the user can access raw USB devices.  Doing this as an unprivileged user isn't a bad idea - especially since you'll be running shell scripts you're probably not too familiar with.  If you are comfortable doing this as root, then you won't need to add that user to to the usb group.

Step 7:  Let's get ready to flash the board.  Become the user you wish to do all the flashing as, and get to your CHIP-tools directory.  Then make sure that the sunxi-tools and current directory are in your PATH:

$ cd CHIP-tools
$ export PATH=".:../sunxi-tools:$PATH"

Then prepare the board for flashing.  Connect GND (pin 39) to FEL (pin 7) on connector U14 (this is one of the two large DIL connectors on the board).  This can be done with a small piece of 24 gauge solid wire, although using a proper solderless breadboard-style jumper would be better.  This jumper places the SoC into FEL mode, and allows all the sunxi-utilities to do their magic.

Then connect the C.H.I.P. to your computer using a micro-USB to standard USB ("USB-B") cable.  The C.H.I.P. will power-on (and be powered by your computer).

Step 8:  Flash the board with the new firmware and fresh copy of Linux

$ ./chip-update-firmware.sh -d -b stable-gui -f

VERY IMPORTANT NOTE:  This will completely wipe-out whatever you had on your CHIP.

The NTC documentation in the web pages referenced above explains the other flashing options that are available.  These may be handy to know if you want to put a different OS onto the C.H.I.P. or use it in text-only mode.  I'm actually thinking of installing Gentoo Linux on the thing at some point, because I loathe systemd (as I said in my previous rant).

If you follow the above procedure, you will see a dialog similar to the following (similar because the scripts already downloaded the firmware during a previous, unsuccessful, run):

$ ./chip-update-firmware.sh -d -b stable-gui -f
waiting for fel..........OK
debian selected
BRANCH = stable-gui
fastboot enabled
ROOTFS_URL=http://opensource.nextthing.co.s3.amazonaws.com/chip/debian/stable-gui/3
BUILD=3
BR_URL=http://opensource.nextthing.co/chip/buildroot/stable/73/images
BR_BUILD=73
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/rootfs.ubi exists... comparing to http://opensource.nextthing.co.s3.amazonaws.com/chip/debian/stable-gui/3/rootfs.ubi
MD5: 201a36e60168e2db514b220024b10cd3
S3_MD5: 201a36e60168e2db514b220024b10cd3
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/sun5i-r8-chip.dtb exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/sun5i-r8-chip.dtb
MD5: 09707eb7aa4709365cc14aa0314e4716
S3_MD5: 09707eb7aa4709365cc14aa0314e4716
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/sunxi-spl.bin exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/sunxi-spl.bin
MD5: 1dd925fa15a21bc60dd2172fe3ac40ad
S3_MD5: 1dd925fa15a21bc60dd2172fe3ac40ad
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/sunxi-spl-with-ecc.bin exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/sunxi-spl-with-ecc.bin
MD5: f96fc379e59673e23b3fc1b99a89f569
S3_MD5: f96fc379e59673e23b3fc1b99a89f569
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/uboot-env.bin exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/uboot-env.bin
MD5: 6f2b79a781f9f490911012ec3aa653e9
S3_MD5: 6f2b79a781f9f490911012ec3aa653e9
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/zImage exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/zImage
MD5: 91614d8d67bd87987ea5bc958e61cfc3
S3_MD5: 91614d8d67bd87987ea5bc958e61cfc3
file already downloaded
/mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware/images/u-boot-dtb.bin exists... comparing to http://opensource.nextthing.co/chip/buildroot/stable/73/images/u-boot-dtb.bin
MD5: a9051ff21d462ecc66d7ee9ded2ebe0a
S3_MD5: a9051ff21d462ecc66d7ee9ded2ebe0a
file already downloaded
fastboot enabled
BUILDROOT_OUTPUT_DIR = /mnt/ed1/CHIP-FLASH/CHIP-tools/.firmware
== preparing images ==
PADDED_SPL_SIZE=0x000000c4
0+1 records in
1+0 records out
4194304 bytes (4.2 MB) copied, 0.00341595 s, 1.2 GB/s
UBOOT_SIZE=0x00400000
PADDED_UBOOT_SIZE=0x400000
0+0 records in
0+0 records out
0 bytes (0 B) copied, 4.2427e-05 s, 0.0 kB/s
echo Configuring Video Mode
Image Name:   flash CHIP
Created:      Mon Jan 18 18:25:40 2016
Image Type:   ARM Linux Script (uncompressed)
Data Size:    1012 Bytes = 0.99 kB = 0.00 MB
Load Address: 00000000
Entry Point:  00000000
Contents:
   Image 0: 1004 Bytes = 0.98 kB = 0.00 MB
== upload the SPL to SRAM and execute it ==
waiting for fel...OK
== upload spl ==
== upload u-boot ==
== upload u-boot script ==
== execute the main u-boot binary ==
== creating sparse image ==
== waiting for fastboot ==
waiting for fastboot.................OK
target reported max download size of 33554432 bytes
sending sparse 'UBI' (30720 KB)...
OKAY [  2.668s]
writing 'UBI'...
OKAY [  2.034s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.632s]
writing 'UBI'...
OKAY [  8.398s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.609s]
writing 'UBI'...
OKAY [  8.398s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.651s]
writing 'UBI'...
OKAY [  8.396s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.596s]
writing 'UBI'...
OKAY [  8.397s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.605s]
writing 'UBI'...
OKAY [  8.395s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.642s]
writing 'UBI'...
OKAY [  8.398s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.652s]
writing 'UBI'...
OKAY [  8.397s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.732s]
writing 'UBI'...
OKAY [  8.398s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.768s]
writing 'UBI'...
OKAY [  8.402s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.804s]
writing 'UBI'...
OKAY [  8.394s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.615s]
writing 'UBI'...
OKAY [  8.395s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.689s]
writing 'UBI'...
OKAY [  8.394s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.635s]
writing 'UBI'...
OKAY [  8.395s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.652s]
writing 'UBI'...
OKAY [  8.391s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.658s]
writing 'UBI'...
OKAY [  8.395s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.681s]
writing 'UBI'...
OKAY [  8.394s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.670s]
writing 'UBI'...
OKAY [  8.394s]
sending sparse 'UBI' (30720 KB)...
OKAY [  2.712s]
writing 'UBI'...
OKAY [  8.398s]
sending sparse 'UBI' (28672 KB)...
OKAY [  2.505s]
writing 'UBI'...
OKAY [  6.671s]
finished. total time: 213.024s
resuming boot...
OKAY [  0.000s]
finished. total time: 0.000s

Step 9:  Once this is complete, you can unplug the C.H.I.P. from your computer, remove the FEL jumper from the connector, and boot the C.H.I.P. normally.  You should see an updated version of Linux as well as a nicely functioning board.

After this procedure, I rebooted the device several times, and it had no more problems booting (as it did previously).  I have not tried to actually do anything with it yet, since I had other things I needed to do this evening.  I expect, though, that I'll be pretty much back to where I was when I first got the device.

Now that I've gotten past this hurdle, I'm hoping to try doing "computer things" with the C.H.I.P., and maybe even some geeky stuff connected up to the thing.  It looks like it could be a nice device, but given how fragile the flash memory is on the unit so far, I'm a bit gun-shy about doing too much tinkering with the OS yet...

Hopefully, this will help someone get through the reflashing process - especially those who are familiar with Linux or who are using Gentoo Linux, and don't want to install a whole VM and development environment.