Monday, August 20, 2018

IBM IMM VSC452 recovery

IMM boot problem

I purchased a x3650 m3 2u IBM server, which on boot would run the fans constantly at maximum RPM and report a failure to connect to IMM. After several minutes (~15min) of reporting a failure to connect to imm the system would boot into UEFI, but no bios settings could be saved except for boot volumes.

I tried to use the BOMC (Bootable Media Creator) to flash a new version of IMM, or at least recover it. However this mechanisim relies on linux detecting a usb ethernet driver to talk to the IMM module. It appeared that the IMM was dead.

After looking around the internals of the server I determined that the IMM was a VSC452 - but there is only a product information datasheet avialable - not useful at all. However from the datasheet it appears that the VSC452 has a mips processor - one that possibly runs linux.

So I binwalked the IMM firmware binary on the BoMC and found uboot, uImage and 2 jffs2 partitions confirming there was mips processor that runs linux.

Next job was to gain access to serial to inspect the boot loader - there were some pads on the motherboarded labeled iBMC_UART_CONN. I probbed these with a mutlimeter and finally a logic analyser to find out which pins are VCC, GND and TX. RX was found by trial and error. The logic analyser also determined the baud rate and the multimeter confirmed it was using 5V. The image below shows some wires I attached to connect to a USB to serial converter.

Firmware

TFTP Server Setup

Extracting firmware

The firmware I used was an older version - ibm_fw_imm_yuooe3c-1.33_linux_32-64.bin

Extract the firmware with binwalk:

binwalk -Me ibm_fw_imm_yuoog6c-1.44_linux_32-64.bin

-e extract
-M recursively extract

Copy files from the extracted bootfs:

cd _ibm_fw_imm_yuoog6c-1.44_linux_32-64.bin.extracted/_ibmc.tgz.extracted/_0.extracted/_bootfs.jffs2.extracted/jffs2-root/fs_1

cp lib/ld-uClibc-0.9.29.so /tftpboot/ubiimages/ld-uClibc.so.0
cp lib/libuClibc-0.9.29.so /tftpboot/ubiimages/libc.so.0
cp bin/ubimkvol /tftpboot/ubiimages/ubimkvol
cp bin/ubirmvol /tftpboot/ubiimages/ubirmvol
cp bin/ubiupdatevol /tftpboot/ubiimages/ubiupdatevol

Copy the bootfs, rootfs and uImage

cd _ibm_fw_imm_yuoog6c-1.44_linux_32-64.bin.extracted/_ibmc.tgz.extracted/_0.extracted/
cp bootfs.jffs2 /tftpboot/ubiimages/
cp rootfs.jffs2 /tftpboot/ubiimages/
cp uImage /tftpboot/ubiimages/
cp uImage /tftpboot/

Create dummy flash_eraseall:

echo "#!/bin/sh" > /tftpboot/ubiimages/flash_eraseall

Serial Port

Serial Port on Motherboard

Between the two usb ports and VGA connector there is an unpopulated connector labeled iBMC_UART_CONN, show below


The pins are as following, in the order from the USB port to the VGA connector:

 ---     123456    0.---.0
|   |   ,'''''',  | |   | |
|   |             |_-----_|

 USB                 VGA

2 - RX
4 - TX
6 - GND




U-Boot

Enable IBMC_FORCE_UPDATE_N (sw3,4 on) then power the unit and "FORCE UPDATE ENABLED" should appear as shown in the output below. This enables tftp recovery by booting a uImage with ramfs.

IMM UBoot 1.1.4 (Dec  1 2011 - 07:36:30)

  -> IMM revision 7 detected
  -> Clock set to 300MHz
  -> Caches enabled
  -> Super Loader
  ->  Initializing BUS 5..DONE
  ->  Attempt #1
  ->    reading i2c..OK
  ->    validating..OK
  ->  Default ver wins [04] >= [04]
  ->   MAGIC#  = 4D
  ->   VERSION = 04
  ->  waiting for DDR init..OK
  -> Available RAM set to 112M
Top of RAM usable for U-Boot at: 87000000
Reserving 169k for U-Boot at: 86fd4000
Reserving 1028k for malloc() at: 86ed3000
Reserving 44 Bytes for Board Info at: 86ed2fd4
Reserving 36 Bytes for Global Data at: 86ed2fb0
Reserving 8k for boot params() at: 86ed0fb0
Stack Pointer at: 86ed0f98
Now running in RAM - U-Boot at: 86fd4000
Flash: 512 kB
  -> Detecting Platform ... PLATFORM_ID = 0x00500002
  -> gpio[ 0x06 ] = ( 1,  8 ) - POWER_STATUS
  -> gpio[ 0x07 ] = ( 2, 11 ) - POWER_CONTROL
  -> gpio[ 0x08 ] = ( 2,  1 ) - POWER_LED
  -> gpio[ 0x13 ] = ( 4,  0 ) - WYVERN_ALERT
  -> gpio[ 0x02 ] = ( 1,  1 ) - HEARTBEAT_LED
  -> gpio[ 0x01 ] = ( 1,  4 ) - HALFROM
  -> gpio[ 0x05 ] = ( 4,  6 ) - FORCEROM_UPDATE
Nand:  Manuf=Hynix ID=0xdc 512 MiB
In:    serial
Out:   serial
Err:   serial
  -> Chip Type=452, Revision=7
  -> IOR start 0x00000000 00008000
  -> SIF5-SIBS was 0x43. Clearing...
  -> IOR end   0x00000000 00008000
  -> Configuring eth1 for MDIO
  -> Configuring eth0 MAC address
  -> Configuring eth1 MAC address
Net:   eth0, eth1 [PRIME]
Hit any key to stop autoboot:  0
Disabling watchdog
!-> FORCE UPDATE ENABLED
 ->Configuring eth0 for NCSI
   [eth0] NCSI: Package found 0
   [eth0] NCSI: Channel found 0
   [eth0] NCSI: Initialization complete
 -> Attempting to TFTP a kernel uImage
  -> Configuring eth1 MAC address
  -> Configuring eth0 MAC address
Using eth1 device
TFTP from server 192.168.70.200; our IP address is 192.168.70.125
Filename 'uImage'.
Load address: 0xa1800000
Loading: T

Break into u-boot by either pressing ctrl-c when u-boot is trying to tftp the image or at the "Hit any key to stop autoboot". At the u-boot command line configure the following:

setenv ipaddr
setenv serverip
setenv bootargs AUTOMTD=1 ETH1= TFTP= BOOTVOL=1
saveenv
resetimm chip

The unit will reset and be prepared to press S to interrupt the kernel boot.

...
!-> FORCE UPDATE ENABLED
 -> Attempting to TFTP a kernel uImage
  -> Configuring eth1 MAC address
  -> Configuring eth0 MAC address
Using eth1 device
TFTP from server 192.168.1.148; our IP address is 192.168.1.33
Filename 'uImage'.
Load address: 0xa1800000
Loading: TT#################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################
done
Bytes transferred = 1828603 (1be6fb hex)
 -> Attempting to boot...## Booting image at a1800000 ...
   Image Name:   IMM Linux-2.6.16.46-377
   Image Type:   MIPS Linux Kernel Image (gzip compressed)
   Data Size:    1828539 Bytes =  1.7 MB
   Load Address: a1000000
   Entry Point:  812fb000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK

Starting kernel ...


IMM LINUX started...
Linux version 2.6.16.46-377 (geeko@buildhost) (gcc version 3.4.4) #2 PREEMPT Thu Dec 21 18:00:53 UTC 2017
CPU revision is: 0001906c
...
Freeing unused kernel memory: 536k freed
Algorithmics/MIPS FPU Emulator v1.5
==> Running /init (stage 0)
 -> Autobooting MTD
 -> Boot to [M]TD, [S]hell or [B]urn NAND?
 -> Drop Shell selected
 -> Dropping to mini-shell. Type 'exit' to restart init
(IMM-minishell) [~] >

At the shell run the following commands:

/etc/init.d/ifs.burn

Then press Y to proceed - this process should fail with error shown below, but the libraries and utilities have been tftped.

UBI Library Error at ubi_mkvol: ioctl returned -1 errno=17

Cannot create volume: File exists
  err=-1
Jan 01 00:01:36Unable to create volumes. Perhaps the device is not large enough?

Run the following to obtain the ubirmvol command and remove all the ubi volumes:

tftp -b 2048 -g -l /bin/ubirmvol -r ubiimages/ubirmvol 192.168.1.148
chmod +x /bin/ubirmvol
ubirmvol -d0 -n1
ubirmvol -d0 -n2
ubirmvol -d0 -n3
ubirmvol -d0 -n4
ubirmvol -d0 -n5
ubirmvol -d0 -n6
ubirmvol -d0 -n7
ubirmvol -d0 -n8

Then reboot and break into u-boot again and run:

setenv bootargs AUTOMTD=1 ETH1= TFTP= BOOTVOL=1
setenv bootvolume=1
savenv
bootbmc

Interrupt the kernel with B this time and press Y to initiate flashing, for example:

> setenv bootvolume=1
Disabling watchdog
> setenv bootargs AUTOMTD=1 ETH1=192.168.1.33 TFTP=192.168.1.148 BOOTVOL1=1
Disabling watchdog
> saveenv
Disabling watchdog
Saving Environment to Flash...
Un-Protected 1 sectors
Erasing Flash...[s_first=7 s_last=7]
Erased 1 sectors
Writing to Flash... [src=0x86ec0b18 addr=0xbe070000 cnt=65536]
done
Protected 1 sectors
> bootbmc
Disabling watchdog
!-> FORCE UPDATE ENABLED
 -> Attempting to TFTP a kernel uImage
  -> Configuring eth1 MAC address
  -> Configuring eth0 MAC address
Using eth1 device
TFTP from server 192.168.1.148; our IP address is 192.168.1.3
Filename 'uImage'.
Load address: 0xa1800000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################
done
Bytes transferred = 1828603 (1be6fb hex)
 -> Attempting to boot...## Booting image at a1800000 ...
   Image Name:   IMM Linux-2.6.16.46-377
   Image Type:   MIPS Linux Kernel Image (gzip compressed)
   Data Size:    1828539 Bytes =  1.7 MB
   Load Address: a1000000
   Entry Point:  812fb000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK

Starting kernel ...


IMM LINUX started...
Linux version 2.6.16.46-377 (geeko@buildhost) (gcc version 3.4.4) #2 PREEMPT Thu Dec 21 18:00:53 UTC 2017
CPU revision is: 0001906c
Determined physical RAM map:
 memory: 003a8000 @ 01000000 (reserved)
 memory: 06458000 @ 013a8000 (usable)
 memory: 00800000 @ 07800000 (reserved)
Built 1 zonelists
Kernel command line: AUTOMTD=1 ETH1=192.168.1.33 TFTP=192.168.1.148 BOOTVOL1=1 FORCEUP=1  MANMODE=1
2 MIPSR2 register sets available
Primary instruction cache 16kB, physically tagged, 2-way, linesize 16 bytes.
Primary data cache 16kB, 2-way, linesize 16 bytes.
Synthesized TLB refill handler (20 instructions).
Synthesized TLB load handler fastpath (32 instructions).
Synthesized TLB store handler fastpath (32 instructions).
Synthesized TLB modify handler fastpath (31 instructions).
PID hash table entries: 512 (order: 9, 8192 bytes)
CPU frequency 300.00 MHz
Using 150.000 MHz high precision timer.
VSC452 general purpose serial console setup!
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 101632k/102752k available (2563k kernel code, 1092k reserved, 484k data, 536k init, 0k highmem)
Security Framework v1.0.0 initialized
Mount-cache hash table entries: 512
Checking for 'wait' instruction...  available.
NET: Registered protocol family 16
JFFS2 version 2.2. (NAND) (SUMMARY)  (C) 2001-2003 Red Hat, Inc.
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
Generic RTC Driver v1.07
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
Kernel Build: 2:ibmc51_2:mcp:2017:12:21:175504:5:1:0:
NAND device: Manufacturer ID: 0xad, Chip ID: 0xdc (Hynix NAND 512MiB 3,3V 8-bit)
busw=0, options=0xc0000001
Scanning device for bad blocks
Bad eraseblock 263 at 0x020e0000
Bad eraseblock 1194 at 0x09540000
Bad eraseblock 1608 at 0x0c900000
Bad eraseblock 2167 at 0x10ee0000
Bad eraseblock 2638 at 0x149c0000
Bad eraseblock 3624 at 0x1c500000
Bad eraseblock 3625 at 0x1c520000
Bad eraseblock 3730 at 0x1d240000
cmdlinepart partition parsing not available
Setting MTD NAND partition from 536870912 to 536870912
Using static partition definition
Creating 1 MTD partitions on "vsc452-nand":
0x00000000-0x20000000 : "UBI"
VSC GPUART driver initialized
ttySC0 at MMIO map 0x0 mem 0xb0200300 (irq = 6) is a vsc452 gpuart
Netfilter messages via NETLINK v0.30.
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 2, 16384 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
ip_tables: (C) 2000-2006 Netfilter Core Team
ipt_recent v0.3.1: Stephen Frost .  http://snowman.net/projects/ipt_recent/
arp_tables: (C) 2002 David S. Miller
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
802.1Q VLAN Support v1.8 Ben Greear
All bugs added by David S. Miller
NET: Registered protocol family 27
Freeing unused kernel memory: 536k freed
Algorithmics/MIPS FPU Emulator v1.5
==> Running /init (stage 0)
 -> Autobooting MTD
 -> Boot to [M]TD, [S]hell or [B]urn NAND?
 -> Burn NAND selected
NAND Preparation Script
-----------------------
This script will erase your NAND flash device and populate it
with UBI images stored on a TFTP server.

This script will format the volumes as follows (customer layout):
  Volume 1:    Kernel                   2.0MiB -   17 blocks
  Volume 2:    Kernel Backup            2.0MiB -   17 blocks
  Volume 3:    Root Filesystem         50.0MiB -  413 blocks
  Volume 4:    Root Filesystem Backup  50.0MiB -  413 blocks
  Volume 5:    R/W FileSystem          40.1MiB -  331 blocks
  Volume 6:    Staging Area           280.1MiB - 2313 blocks
  Volume 7:    Boot Filesystem          2.0MiB -   17 blocks
  Volume 8:    Boot Filesystem Backup   2.0MiB -   17 blocks
               PreLayout Block Pool    19.1MiB -  158 blocks
               PostLayout Block Pool   48.5MiB -  400 blocks
               3.125% UBI Overhead     16.0MiB -  n/a
                                       -------   -----------
                    (536870912 bytes) 512.0MiB   4096 blocks


 -> Proceed?(Y/N) y
Jan 01 00:00:02 -> Loading ethernet kernel module
Jan 01 00:00:08 -> Looking for TFTP server (192.168.1.148)
Jan 01 00:00:10 -> Found TFTP server on host 192.168.1.148
Jan 01 00:00:10 -> Looking for remote file (ld-uClibc.so.0)
Jan 01 00:00:10 -> File found at TFTP location 192.168.1.148:ubiimages/ld-uClibc.so.0)
Jan 01 00:00:10 -> Looking for remote file (libc.so.0)
Jan 01 00:00:11 -> File found at TFTP location 192.168.1.148:ubiimages/libc.so.0)
Jan 01 00:00:11 -> Looking for remote file (ubimkvol)
Jan 01 00:00:11 -> File found at TFTP location 192.168.1.148:ubiimages/ubimkvol)
Jan 01 00:00:11 -> Looking for remote file (ubiupdatevol)
Jan 01 00:00:11 -> File found at TFTP location 192.168.1.148:ubiimages/ubiupdatevol)
Jan 01 00:00:11 -> Looking for remote file (flash_eraseall)
Jan 01 00:00:11 -> File found at TFTP location 192.168.1.148:ubiimages/flash_eraseall)
Jan 01 00:00:11 -> All common files downloaded
Jan 01 00:00:11 -> Looking for remote file (uImage)
Jan 01 00:00:13 -> File found at TFTP location 192.168.1.148:ubiimages/uImage)
Jan 01 00:00:13 -> Looking for remote file (bootfs.jffs2)
Jan 01 00:00:15 -> File found at TFTP location 192.168.1.148:ubiimages/bootfs.jffs2)
Jan 01 00:00:15 -> Looking for remote file (rootfs.jffs2)
Jan 01 00:01:13 -> File found at TFTP location 192.168.1.148:ubiimages/rootfs.jffs2)
 -> All burn files downloaded
Jan 01 00:01:13 -> All common prechecks passed
 -> All burn prechecks passed
 -> Preparing MTD for images
Jan 01 00:01:13 -> Erasing NAND flash device
dummy!
Jan 01 00:01:13 -> Loading ubi kernel module
Jan 01 00:01:18 -> Checking primary ubi device nodes
 -> Creating volumes
Jan 01 00:01:18 -> Creating all volume device nodes
 -> Populated all volumes
Operation complete.
We will reboot shortly. On next reboot, use the 'bootbmc'command inside UBoot
When prompted, select [M]TD mount.
Rebooting in 5 seconds...
Restarting system.

Wait for the system to restart and then break into uboot and power down the unit and turn off IBMC_FORCE_UPDATE_N (sw3,4 off).










Posted by at No comments:

Friday, January 31, 2014

OpenWRT for WEMO - Part 3 - Cross compilation and remote debug with Eclipse

Being able to compile and remotely debug applications on the WEMO is very useful in creating custom applications for remotely switching the power, or adding functionality to the WEMO. For remote debugging the WEMO must be connected to the wifi network that the cross compilation machine is on and the machine must be able to ssh into the WEMO. 

Here are some example settings to get the WEMO connected to a typical WPA-PSK wifi network. These settings can be modified over the serial terminal.
  • Edit /etc/config/network with vi and comment every line with # then add the follwing.
     
    config interface 'lan'
        option ifname 'wlan0'
        option proto 'dhcp'
        option hostname 'WEMO'
     
  • Edit /etc/config/wireless with vi and change it to the following, note change the SSID and password to that of your Wifi network.

    config wifi-iface
        option device 'radio0'
        option network 'lan'
        option mode 'sta'
        option ssid 'SSID'
        option encryption 'psk2'
        option key 'password'
     
  • Reboot the router and check over the serial terminal that the wlan0 interface has been assigned an ip address.

Using the custom compiled firmware from part 1 the required packages for remote debugging will be installed. If using the base image use the following command to install the required packages.  
opkg update
opkg install gdb-server openssh-sftp-server

The following steps setup Eclipse to cross compile a simple c project.
 
  1. Start Eclipse and navigate to "Help" menu and select "Install New Software...". In the install wizard select "--All Available Sites--" from the drop down menu next to "Work with:" at the top. This will load all available software packages that can be install in Eclipse and will take while.
     
  2. If your version of eclipse does not have CDT installed use the guide here to install it. Navigate to the "Mobile and Device Development" in the tree view and select "C/C++ GCC Cross Compiler Support", "Remote System Explorer End-User Runtime", and "C/C++ Remote Launch" as shown below.


    Click "Next >"  and follow the prompts in the wizard to install the packages. Eclipse may need to restart afterwards.
     
  3. In this example we will be creating a "C Project" that will cross compile C code only. Select "File" then "New" and finally "Project..." to bring up the "New Project" wizard. Choose the "C/C++" option in the tree view and then select "C Project" as shown below.


    Click "Next >" to proceed.
     
  4.  Select an "Empty Project" from "Executable" under "Project type:" and "Cross GCC" under "Toolchians:" as shown below.


    Choose a name for the project and enter it at the top then click "Next >" to proceed.
     
  5. In the final stage of the wizard (the "Finnish button" shown appear) click the "Advanced Settings" button. This will bring up a dialogue to edit the project settings, which can be accessed by right clicking on the project and selecting "Properties". The next steps deal with this dialogue and show the parameters that need to be added to configure the cross compiler.
  6. The staging directory in the openwrt folder contains the cross compiler and gdb for remote debugging. The first step is to configure the cross compiler, and this is under "C/C++ Build" then "Settings" on the left. Choose "Cross Settings" as shown in the image below.


    The toolchains directory depends on the architecture and gcc version. Locate the toolchain directory by looking in the staging directory of openwrt and set it to the path. The prefix is an appended string for the cross compilation tools such as gcc, this can be found by looking in the bin directory of the toolchain. Note all directories must be absolute and directories like "~/openwrt/..." will not work.
     
  7. The include directories can be specified to enable better code completion by selecting Includes as shown in the image below.


    Select the add button (upper right in the image) and locate the include directory in the toolchain directory.
     
  8. An environment variable "STAGING_DIR" is required by the cross compiler and can be set by navigating to "C/C++ Build" then "Environment" on the left as shown below.


    To add the environment variable click the "Add" and set the name as "STAGING_DIR" and the value as the path to the OpenWRT staging directory.
     
  9. Finnaly click "OK" to save the project settings and finally "Finish" to create the project.
  10. To test the cross compiler create a "main.c" file in the project by right clicking the project and selecting "New" then "File". Use the following code as a simple hello world.
     #include

    int main()
    {
        printf("hello world\r\n");
        return 0;
    }
     
  11. Save the main.c file then right click the project and select build. The output of the build (in the "Console" tab down the bottom) show a successful build, for example:

    **** Build of configuration Debug for project hello_world ****

    make all
    Building file: ../main.c
    Invoking: Cross GCC Compiler
    mipsel-openwrt-linux-gcc -I/home/sean/openwrt/openwrt/staging_dir/toolchain-mipsel_24kec+dsp_gcc-4.6-linaro_uClibc-0.9.33.2/include -O0 -g3 -Wall -c -fmessage-length=0 -MMD -MP -MF"main.d" -MT"main.d" -o "main.o" "../main.c"
    Finished building: ../main.c

    Building target: hello_world
    Invoking: Cross GCC Linker
    mipsel-openwrt-linux-gcc  -o "hello_world"  ./main.o  
    Finished building target: hello_world


    **** Build Finished ****

    If there are any problems with the build, check to ensure the paths to the tools are correct. If the path to the build tools requires changing then change the "PATH" environment variable, as changing the setting in step 6 will not update the environment variable.
To remotely debug a cross compiled project use the steps below.
  1. Ensure the "main.c" file in cross compiled project is selected in the "Project Explorer" then open the debug configuration by clicking the arrow next to the debug button and then selection "Debug Configurations..." as shown below.


    This will open the Debug configuration dialouge.
     
  2. Create a new "C/C++ Remote Application" configuration by clicking the button shown in the image below.


    This will create a new debug configuration in the left pane.
  3. Set the "Remote Absolute File Path for C/C++ Application:" to a simple path such as /tmp/hello_world. It is better to use the tmp directory as it resides in ram making upload faster and decreases wear on the flash.
     
  4. Click the "New..." button next to the combo box for "Connection:" to add a connection to the WEMO. Select a "Linux" connection in the wizard as shown below, then click "Next".


  5. Give the the IP address of the WEMO on your WiFi network for the "Host Name:", and optionally a friendly "Connection name:" as shown below.


    Do not click "Finnish", instead click "Next".
     
  6. Select "ssh.files" from the "Configuration" check list as shown below.


    Proceed by clicking "Next".
     
  7. Select "processes.shell.linux" as shown below then click "Next".

     
  8. Select "ssh.shells" as shown below then "Finish" to complete the wizard.


    This creates a new option in the combo box next to "Connection:" for the WEMO in the debug configuration.
  9. Select the newly created connection from the "Connection:" comb box. Select "Enable auto build" to enable building before debugging.
  10. Navigate to the Debugger tab (at the top), click "Browse..." next to the "GDB debugger" option and select the cross compiled GDB debugger in the toolchains bin dierctory. The GDB debugger is cross compiled as a part of the toolchain, and is located in the bin folder of the toolchain directory used earlier. The GDB executable is prefixed like all the other tools.
  11. Click the "Debug" button in the right hand corner to intiate a remote debug session. A prompt for the ssh login will appear if the connection is successful. After logging in the executable will be uploaded and debugging session started. The program should stop on a break point on the first line of code.
The console tab provides access to the remote shell session but must be selected by clicking the arrow next to the console button and selecting the remote shell option as shown below.

If you step through the hello world program, hello world will appear in this console.

If an "Error during file upload" occurs this can mean an existing debug session has not terminated correctly. This requires the termination of the gdbserver process, which can be done in the "Remote Systems" tab. To enable the "Remote Systems" tab, open the "Window" menu and select "Show View" then "Other...". In the dialogue use the search button to find the "Remote Systems" tab. The gbdserver process can be terminated by selecting the connection to the WEMO in the "Remote Systems" tab, then selecting processes. Find the gdbserver process in the treeview, right click, and select "Kill..." as shown below.


That completes a guide to cross compiling and remote debugging in eclipse. Next I will show how to interface with the gpio pins to switch the relay in the WEMO.
Posted by at 3 comments:
Labels: , , , , ,

OpenWRT for WEMO - Part 2 - Flashing Firmware

So the WEMO arrived, and the first thing I did was set it up with my iPhone. After checking it worked I voided the warranty to attach serial lines. The serial port is used to issue commands to flash firmware onto the WEMO and requires soldiering onto the board. There are pads for RX and TX, but the pad for ground proved tricky to heat sufficiently so I used the ground pin of the header in fear of damaging the board. There is also conveniently placed via to put the ground wire through, I refrained from soldering to the via for the same reason as the ground pad.


The location of the pads and various other information about the hardware can be found here and here. I attached the ground, rx, and tx to Sparkfun FTDI XBee explorer to give access to the serial terminal. Powering on the WEMO gave a very verbose boot log.

The flash memory partitioning on the WEMO can be determined from the boot messages.

Creating 10 MTD partitions on "raspi":
0x00000000-0x00050000 : "uboot"
0x00050000-0x007c0000 : "A - Kernel and Rootfs"
0x00150000-0x007c0000 : "A - Rootfs"
0x007c0000-0x00f30000 : "B - Kernel and Rootfs"
0x008c0000-0x00f30000 : "B - Rootfs"
0x00fe0000-0x00ff0000 : "Nvram"
0x00ff0000-0x01000000 : "User_Factory"
0x00040000-0x00050000 : "Factory"
0x00f30000-0x00fd0000 : "Belkin_settings"
0x00030000-0x00040000 : "Uboot_env"

The stock firmware allows for updating from Belkin, so to increase reliability there are actually 2 firmware images stored in flash. From the partitions above you can see where they reside in flash. Uboot uses the environment variables "bootstate" and "check_boot" for switching between which images to boot from (more info can be found on the OpenWRT site).

When I first uploaded the firmware onto the WEMO I assumed that OpenWRT would either override this or nicely work with their redundancy scheme. Unfortunately it didn't quite work out because the partitions OpenWRT expected to see weren't there as it was put into the "B - Kerenel and Rootfs" partition.

Using the fwupgrade tool in the stock firmware I flashed my compiled firmware image in, but at boot it would not mount the rootfs partition. Setting the uboot "bootarg" variable with different mtdblock numbers did not remedy the situation. So I booted back into the stock firmware (changed "bootstate" to 0) and dd'ed the stock firmware to overwrite openwrt using: 

dd if=/dev/mtdblock1 of=/dev/mtdblock3
 
This overwrites the kernel and rootfs. The intial state of the flash memory appears to have identical contents of a and b firmware partitions. To flash the OpenWRT firmware requires that a stock firmware version of less than WeMo_WW_2.00.2176.PVT. To find the firmware version use this command in the serial terminal

cat /etc/ver.txt

 The contents of my ver.txt is:
 
02/19/2012 06:35:50 AM
WeMo_WW_1.01.1105.PVT

If you have an upgraded WEMO it may be possible to still boot into the stock firmware, you can check if the stock firmware is in the b partition by using step 4 and 5 below.

To write the OpenWRT firmware to the WEMO use the following steps:

  1. Ensure the WEMO is connected to your WIFI network by setting it up, DO NOT UPGRADE THE FIRMWARE if prompted.

  2. This step is optional - if you want to used a custom compiled image. Install apache or any other web server. This is needed to download the firmware image. Place the compiled firmware image in the in root web server directory and ensure it is accessible.

  3. Open a terminal for the serial port, I used PuTTy, the buad rate is 57600.
     
  4. While in the terminal hold down "4" and then boot the WEMO, uboot only has a 1 second wait time.
     
  5. In the uboot command line set it to boot from the b firmware image using:
     
    set bootstate 2
set boot_check 0
set bootargs 'console=ttys1,57600n root=/dev/mtdblock4'
saveenv
boot
    After the boot command the WEMO will reboot into the stock firmware image from the b firmware partition.
     
  6. Wait for about 10 seconds for a prompt, the serial terminal is the default output for some of the log messages so messages will appear sporadically. To stop these log messages use:
     
    killall sh
killall wemoAPP

    This kills the Belkin applications that enable remote switching and log to the serial port.
     
  7. Now we can download the firmware image into the tmp (this is for a firmware image from openwrt.org, change the url accordingly):
     
    cd /tmp
wget http://downloads.openwrt.org/snapshots/trunk/ramips/openwrt-ramips-rt305x-belkinf7c027-squashfs-sysupgrade.bin

    The tmp directory resides in RAM, all other directories are in flash so the image will not fit. I have given the URL to the trunk snapshot on openwrt.org, if you are using a custom image give the url to webserver that has the image.
     
  8. The image is now in the tmp directory and it can be flashed using:
     
    fwupgrade openwrt-ramips-rt305x-belkinf7c027-squashfs-sysupgrade.bin
     
  9. Reboot the router using the "reboot" command and then hold "4" to enter the uboot command menu. The "bootstate" and "boot_check" variable need to be set in order to prevent uboot reverting back to the stock firmware.
     
    set bootstate 0 
set boot_check 0
saveenv
boot

    The WEMO will now reboot and hopefully boot into OpenWRT. The nice thing is that the stock firmware can be reverted to by changing the boot variables using step 5.
In the next part I will show how to use Eclipse to compile and remotely debug software for the WEMO using the toolchain compiled in part 1.
Posted by at No comments:
Labels: , , ,

Thursday, January 30, 2014

OpenWRT for WEMO - Part 1 - Compiling firmware

While waiting for a Belkin WEMO to arrive in the mail, I decided to at least get the firmware and development tools ready for its arrival. The Belkin WEMO is a power socket switch that can be turned on and off via the internet. It runs an embedded form of linux, with 16 Mb of storage and 32 Mb of ram. The embedded processor it uses (RT5350F) is part of the ralink mips embedded processor line for networking devices such as routers. OpenWRT is embedded linux for routers and many ARM and MIPS based devices. The ralink MIPS processor is supported by OpenWRT, so I will show how to install OpenWRT on the router and develop software for it.  This will enable development of custom applications to run on the device for switching appliances not restricted to Belkins proprietary software. Alternatively you could use the uPnP protocol with the stock firmware to control the unit (here), but that's a messy solution that requires a computer!

There are some prerequisites for putting OpenWRT on the WEMO, which are:

  • You must have Linux, preferably Ubuntu
  • The WEMO must be running stock firmware (pre 2.00.2176), due to an exploit Belkin locked access to the device (see here).

The following steps show how to create the toolchain for cross compilation, create tools for remote debugging, and create the firmware image for the WEMO.

  1. Install the latest build tools for linux.

    sudo apt-get install subversion build-essential libncurses5-dev
sudo apt-get install git-core python2.7-dev
     
  2. Checkout the latest OpenWRT.
     
    mkdir ~/openwrt
cd ~/openwrt
git clone git://git.openwrt.org/openwrt.git
     
  3. Update and download the feeds, the feeds are source code for building packages to install in the firmware image.
     
    cd ~/openwrt/openwrt
./scripts/feeds update -a
./scripts/feeds install -a
  4. Start the menu configuration for OpenWRT, this will check for dependencies required for cross compilation.
     
    make menuconfig
  5. The menu configuration configures which cross compilation tools are to be compiled for creating a firmware image for a specific platform. For the Belking WEMO select the Target System as "Ralink RT288x/RT3xxx" and select the Subtarget as "RT3x5x/RT5350 based boards"
     
  6. This step enables options to build a toolchain for cross compiling and tools for remote debugging in eclipse (tutorial how to do this here). To build a toolchain and remote debugging tools select the following options:
     
    • Enable "Advanced configuration options (for developers)" (press space  bar), and from this menu (press enter to open once enabled) select "Toolchain Options --->" and ensure "Build gdb (NEW)" is enabled.
    • Back in the main menu, enable "Build the OpenWRT based Toolchain"
    • Select the "Development --->" menu and enable the "gdbserver" option, To select the option press space bar twice until an * appears next to the option, when M appears next to an option the package will be built but not installed in the firmware image.
    • Back to the main menu, then select "Network --->", "SSH --->" and enable the package "openssh-sftp-server"
     
    These options will allow for compilation and remote debugging using Eclipse, guide will be [here]
     
  7. Exit menu configuration and when prompted save the configuration.
     
  8. Begin the build processing with:
     
    make
     
    This can take several hours, if there are problems use the option V=99 to get a more verbrose output. 

This completes the build of the firmware image, which can be found in:
   
~/openwrt/openwrt/bin/ramips/openwrt-ramips-rt305x-belkinf7c027-squashfs-sysupgrade.bin

The next step (here) is to connect serial lines to the Wemo, prepare the uboot environment, and finally flash the firmware image.

Posted by at 1 comment:
Labels: , , ,

Tuesday, September 10, 2013

OpenWRT based Video Recording System - Part 2 - Software setup for OpenWRT

Fromatting SD card with EXT2

In order to use the SD card with OpenWRT it must be formatted with the EXT2 filesystem. For flash based storage devices EXT2 is preferred as it reduces the amount of writes performed, as it is not a journaled filesystem. Flash devices have a limited number of write cycles, and while this is limit is very high (millions or more) reducing the number of writes will increase the lifespan of the device. A non journaled filesystem essentially means that it does not track changes to files, which results in less writes performed.

These are the steps to format an SD card using Ubuntu:
  1. Check to see where SD card is mounted.

    root@ubuntu:~# mount
    ...
    /dev/sdb1 on /media/sean/9016-4EF8 type vfat
     
  2. Unmount the SD card.

    root@ubuntu:~# umount /dev/sdb1
     
  3. Run fdisk to remove existing partition and create new partition for EXT2.

    root@ubuntu:~# fdisk /dev/sdb

    Command (m for help): d
    Selected partition 1

    Command (m for help): n

    Partition type:
       p   primary (0 primary, 0 extended, 4 free)
       e   extended
    Select (default p): p
    Partition number (1-4, default 1): 1
    First sector (2048-15407103, default 2048):
    Using default value 2048
    Last sector, +sectors or +size{K,M,G} (2048-15407103, default 15407103):
    Using default value 15407103

    Command (m for help): p

    Disk /dev/sdb: 7888 MB, 7888437248 bytes
    13 heads, 13 sectors/track, 91166 cylinders, total 15407104 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000

       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1            2048    15407103     7702528   83  Linux

    Command (m for help): w
    The partition table has been altered!

    Calling ioctl() to re-read partition table.
    Syncing disks.
     
  4. Finally create the EXT2 file-system on the new partition.

    root@ubuntu:~# mkfs.ext2 /dev/sdb1
    ...
    Allocating group tables: done                          
    Writing inode tables: done                          
    Writing superblocks and filesystem accounting information: done
     
The SD Card is now ready for use in OpenWRT. The next steps are to install OpenWRT on the router, and then transfer the OpenWRT filesystem from the internal flash memory to the SD card. Finally we can install the packages needed to capture video from the USB webcam.

OpenWRT configuration on MR3020

 The firmware used on the MR3020 can be obtained from:

 MR3020 OpenWRT 12.09 r36088 for sysupgrade
MR3020 OpenWRT 12.09 r36088 for factory upgrade

 The first link is the firmware image for a sysupgrade using OpenWRT, and the second link is the firmware image that can be used with the factory firmware update.

 The following steps will configure OpenWRT to use the SD card as the overlay. The overlay is where any changes to the file system are stored, as the original file system is read only.
  1. Update opkg repositories.
     
    opkg update
     
  2. Install the kmod-usb-storage, kmod-fs-ext4 and block-mount packages to enabled usb flash drive support.
     
    opkg install kmod-usb-storage kmod-fs-ext4 block-mount
     
  3. Create a directory to mount the USB drive on, and mount it.yes
    mkdir /mnt/sda1
    mount /dev/sda1 /mnt/sda1
     
  4. Copy the contents of the existing overlay to the flash drive. 
         
    tar -C /overlay -cvf - . | tar -C /mnt/sda1 -xf -
     
  5. Edit the file /etc/config/fstab and add the following.

    config mount
            option target   /overlay
            option device   /dev/sda1
            option fstype   ext2
            option options  rw,sync
            option enabled  1
            option enabled_fsck 0
     
  6. Reboot.
  7. Check to see if it has mounted correctly by looking at the free space on the root.
     
    OpenWrt:~# df -h
    Filesystem                Size      Used Available Use% Mounted on
    rootfs                    7.2G      3.3M      6.8G   0% /
    /dev/root                 2.0M      2.0M         0 100% /rom
    tmpfs                    14.3M     72.0K     14.2M   0% /tmp
    tmpfs                   512.0K         0    512.0K   0% /dev
    /dev/sda1                 7.2G      3.3M      6.8G   0% /overlay
    overlayfs:/overlay        7.2G      3.3M      6.8G   0% /
     
Creating a swap file increases the stability of the system as capturing video is memory intensive and may exhaust the limited RAM. Also log files may become large if the system isn't rebooted for long periods, they are stored in a temporary directory which is in RAM. The following steps setup a swap file.
  1. Create the swap file using dd, the count is the number of kilobyes as the blocksize (bs) is a kilobyte (1024 bytes). For a count of 65536 a 64 MB swap will be created. Note this will take some time.

    dd if=/dev/zero of=/swapfile bs=1024 count=65536
     
  2. The file needs to be formatted as a swap file using mkswap.

    mkswap /swapfile
     
  3. The swapfile needs to be mounted by the fstab, so edit /etc/config/fstab and add the following.

    config swap
            option device   /swapfile
            option enabled  1
     
  4. Restart fstab to mount and enable the swap file.

    /etc/init.d/fstab restart
     
  5. Check the swap is working using free. You should see 65532 under the total column for the Swap row.

    # free
                 total         used         free       shared      buffers
    Mem:         29212        27856         1356            0         1216
    -/+ buffers:              26640         2572
    Swap:        65532            0        65532
     
The USB flash drive has now been configured as the overlay and with a swapfile.

Logitech C170 Webcam configuration for OpenWRT

 The next stage is installing packages and configuring the Webcam, which is as following.
  1. Install the packages ffmpeg, and kmod-video-uvc, the Logitech C170 webcam uses the usb video class (uvc) drivers and ffmpeg is used to encode videos.
         
    opkg update
    opkg install ffmpeg kmod-video-uvc
     
  2. Plug the C170 in and check that the drivers work by looking at the dev directory, it should contain video0.
  3. "Motion" is a software package that will capture videos or images when motion has been detected from a video source (the webcam in our case). The standard motion package that was compiled for OpenWRT does not have ffmpeg support, so I compiled a custom version. This version can be obtained here. To install it on the MR3020 do the following.
         
    wget http://228899seankelly.googlecode.com/svn/trunk/ar71xx/motion/motion_20120605-224837-2_ar71xx.ipk
    opkg install motion_20120605-224837-2_ar71xx.ipk
     
  4. Motion has a configuration file in /etc/motion.conf, to capture video from the C170 webcam the following options were set.

    The pixel format of the C170 webcam is MJPEG which corresponds to a value of 8 for the v4l2_palette option:

    v4l2_palette 8

    The size of the image to be captured is 320x240 so this is given by width and height. 320x240 lowers the strain of motion detection.

    width 320
    height 240

    Decreasing the frame rate to less than 5 fps will also reduce the processing required. For my setup I found 3 fps was adequate to detect and record motion, I set it using the following.

    framerate 3

    To detect motion a threshold is required, this is the number of pixels that must change in the picture for motion to be present. For the 320x240 image size and a wide angle camera about 500 to 1000 pixels should be adequate. To set the number of pixels the threshold is set as following.

    threshold 1000

    De-speckling of the motion to produce better defined bounds for the motion can be turned off to further reduce the processing required. To turn of the filters that do this comment out the line below using ";"

    ;despeckle_filter EedDl

    In order to filter out some noise the minimum number of consecutive frames with motion can be set. This stops random noise from cause a motion event. The minimum number of frames is set as following, I used 2 frames as at 3 fps this is 0.6 of second of motion is required to start a motion event.

    minimum_motion_frames 2

    In order to create better video captures a number of frames after and before the motion event can be saved as well to create a more fluid video. The pre_capture and post_capture variables specify the number of frames to capture before and after and event. I set both to 2 frames, as following.

    pre_capture 2
    post_capture 2

    The goal is to use ffmpeg to encode videos so we need to disable image output by setting the following.

    output_pictures off

    The default bitrate for ffmpeg is high for lower resolutions so lowering it to between 200000 bps to 300000 bps should be ok, to do this change the following.

    ffmpeg_bps 200000

    To enable playback in a browser, ffmpeg must encode videos in a compatible format such as a flash swf. To use the swf format set the following.

    ffmpeg_video_codec swf

    The last setting is the directory to store the videos in, to give access to the videos from the inbuilt http server they can be stored in the /www folder using the variable as following.

    target_dir /www/cam
This completes the setup for motion to capture video from the C170 webcam. The files will be output to to the cam directory in the uhttp deamons www directory, which gives access to them from the web. 
Posted by at 8 comments:
Labels: , ,
Subscribe to: Posts (Atom)