CubieBoard Forum

Cubieboard itself => Hardware => Cubieboard v3 Hardware (Cubietruck) => Topic started by: phelum on February 07, 2016, 06:11:40 am

Title: A10/A20 NAND boot block access
Post by: phelum on February 07, 2016, 06:11:40 am
Hi,

Recently I modified the sunxi_nand driver in my 3.4.103 kernel to provide boot block access.  The first seven blocks in the NAND chip are used to store the boot programs required to enable an A10/A20 board to boot from NAND.  The first program is called boot0 and is stored in block 0 with a duplicate in block 1.  The second program is boot1 and is stored in block 2 with duplicates in blocks 3 - 6.

The modifications in the driver are included when CONFIG_SUNXI_NAND_BOOT_BLOCK_ACCESS is selected in the kernel config options.  This option is only available when SUNXI_NAND = 'm'.  The reason for this is that the special access functions are only available when the driver is being loaded.  The driver is loaded by a modprobe command and optional tail text in this command invokes the boot block read or burn functions.

Assuming that /opt exists and is the wanted file path,
To read blocks: modprobe sunxi-nand save_path=/opt
To burn blocks: modprobe sunxi-nand src_path=/opt

The files created from a read function are boot0_0 and boot0_1 (boot0 from blocks 0 and 1) and block_2 through block_6 (boot1 from blocks 2 through 6).  These files are truncated to just contain boot0 or boot1 if a valid file header is found.

The files required for a burn function are called boot0 and boot1.

I modified this driver because my bootfix program can't load boards with new NAND chips.  This is because bootfix uses some closed-source Allwinner programs and these don't work if the NAND chip ID is not included in the relevant tables.  This restriction also applies to Livesuit image files (particularly those for A10 SoCs).  For these new boards I also had to modify boot0 and boot1 and compile a new u-boot.bin.  Developing a NAND boot system for a new board can be a tedious task.

A kernel with this driver is of limited value because it can't be installed into NAND.  But it is useful if you'd like to boot a blank board and load the boot blocks and then partition and load the rest of NAND.  It was designed for this purpose and is also good if you'd like to extract the boot programs from a working system.

The driver is available at http://phelum.net/temp/sunxi_nand.tar.gz (http://phelum.net/temp/sunxi_nand.tar.gz).  This archive contains the replacement Kconfig and also all the sunxi_nand files.  The modified files are:

modified:   drivers/block/Kconfig
modified:   drivers/block/sunxi_nand/nfc/nfc_r.c
modified:   drivers/block/sunxi_nand/nfd/nand_blk.c
modified:   drivers/block/sunxi_nand/src/include/nand_drv_cfg.h
modified:   drivers/block/sunxi_nand/src/physic/nand_simple_r.c

All patches in the modified files are headed with #ifdef CONFIG_SUNXI_NAND_BOOT_BLOCK_ACCESS.  Most of the changes are in nand_blk.c.  The small modifications to nfc_r.c and nand_simple_r.c load the special randomizer seed and NFC_SEQ flag required for boot0 access with some NAND chips.

If anybody is interested in this work I can create a git repository and also provide details about customising boot0 for a board.  It needs the correct DRAM and NAND specs.  Normally these details are loaded by Livesuit when downloading a system.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: arthur on April 26, 2016, 02:26:28 pm
Hi Steven,

Thanks for these modifications ! You seems to very well understand the underlying NAND boot process from allwinner. I must admit that this is kind of a black box for me.

Where did you get all these information about boot0 location and boot1 duplicates etc.? Do you have some kind of documentation?

I'm using these tools to make my NAND img: https://github.com/ozgur-keles/livesuit-image-creator

But the generated image has 2 main issues:

1) On the new board freshly flashed if I execute nand-part (see output below), all partitions shown are OK and the size of /dev/nanda is 32768*512 Bytes (=16384KiB). But if I execute df, the vfat filesystem size on this same boot partition has a size of 127.7M. I think that tools in livesuit-image-creator are using some kind of preset values that are not correct in my specific use-case. But most of these tools are not open-source, so I'm not able to set my custom values. What tools are you using to pack your NAND image?

2) Please find below my nand-part output:

Code: [Select]
check partition table copy 0: mbr: version 0x00000200, magic softw411
OK
check partition table copy 1: mbr: version 0x00000200, magic softw411
OK
check partition table copy 2: mbr: version 0x00000200, magic softw411
OK
check partition table copy 3: mbr: version 0x00000200, magic softw411
OK
mbr: version 0x00000200, magic softw411
3 partitions
partition  1: class =         DISK, name =   bootloader, partition start =    32768, partition size =    32768 user_type=0
partition  2: class =         DISK, name =       rootfs, partition start =    65536, partition size =   786432 user_type=0
partition  3: class =         DISK, name =        UDISK, partition start =   851968, partition size =    38912 user_type=0

And now the content of /proc/partitions

Code: [Select]
  93        0     498688 nand
  93        1      16384 nanda
  93        2     393216 nandb
  93        3      72704 nandc

The third partition size seems wrong because nand-part was saying that it was 38912*512=19,9MB.

Did you notice this kind of behavior with your tools?

Thanks for your help !

Arthur
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 26, 2016, 04:44:29 pm
Hi Arthur,

1) On the new board freshly flashed if I execute nand-part (see output below), all partitions shown are OK and the size of /dev/nanda is 32768*512 Bytes (=16384KiB). But if I execute df, the vfat filesystem size on this same boot partition has a size of 127.7M. I think that tools in livesuit-image-creator are using some kind of preset values that are not correct in my specific use-case. But most of these tools are not open-source, so I'm not able to set my custom values. What tools are you using to pack your NAND image?

I'm not using anything to pack an image.  Livesuit seems to do more than most people realise and it changes things to suit the target machine.  But if the target machine contains some unknowns (e.g. DRAM chip) then Livesuit fails.  The work I did was to load some custom boards and I had to do things like load Boot0 with the DRAM info myself and manually create the first image in NAND.  But once this was done it is possible to copy the NAND partitions to an SD card.  If the card has a kernel with the modified NAND driver and a script it can be used to automatically load other boards.  Good for a production line.

The third partition size seems wrong because nand-part was saying that it was 38912*512=19,9MB.

Did you notice this kind of behavior with your tools?

There is something wrong somewhere in the system where the size of the last partition is incorrect.  I put a dummy partition (small as possible) at the end and never touch it.  I think it's a nand-part problem but I'm not sure.

I've seen problems when the nand-part specs used to partition your NAND had a final partition size of 0 which means use all available.  But in your case it looks like the problem is elsewhere (maybe the driver).  Have you formatted nandc and if so what size does it report ?  Frankly, since it's such a tiny and useless partition I'd leave it alone.  It seems to do the job of my dummy partition which is to cause the size of the previous partition to be reported correctly.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: arthur on April 27, 2016, 08:17:43 am
Hi Steven,

Thanks for your quick feedback,

Livesuit seems to do more than most people realise and it changes things to suit the target machine.  But if the target machine contains some unknowns (e.g. DRAM chip) then Livesuit fails.  The work I did was to load some custom boards and I had to do things like load Boot0 with the DRAM info myself and manually create the first image in NAND. 

You are right, Livesuit was just a binary image flasher for me. I'm using PhoenixTool which is Livesuit 2.0 according to the sunxi wiki. Which of these tool are you using? Because I can only upload an image to the NAND on a board previously booted in FEL mode. Or maybe I missed something big  :o

You made your own NAND image step by step, but how did you know that you needed to put boot0 at block 0 and a duplicate at block 1? Did you get these information from the source code?

I found boot0 and boot1 source code here: https://github.com/allwinner-zh/bootloader/tree/master/basic_loader (https://github.com/allwinner-zh/bootloader/tree/master/basic_loader) but it seems tough to understand all of this from the source code.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 28, 2016, 12:09:50 am
Hi Arthur,

I found some of the boot block info in a full release (Draco H3 I think) and the rest from bits and pieces that are around.  I've never seen a comprehensive document anywhere.

All of my work in this area has been for A10/A20 boards and it's all a bit old and irrelevant now.

Is there any particular area you're interested in ?  Or are you just looking for an explanation about the size of your last partition ?  After I said yesterday that the mismatch is expected I checked some of my boards here and they don't have the problem.  Maybe it is a problem with nand-part.  But since your last partition is tiny and can't really be used then perhaps the size mismatch isn't a problem anyway.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: arthur on April 28, 2016, 04:12:34 am
Hi Steven,

Ok thanks, maybe my issue is coming from my misuse of the packer tool.

In order to analyze my generated image, I'm using imgRePacker to unpack the image. I noticed that my sys_partition.fex was not correct:

Code: [Select]
[mbr]
size = 16384

[partition_start]

;------------------------------>/nanda
[partition]
    name         = bootloader
    size         = 32768
    downloadfile = "bootloader.fex"
    verify       = 1

;------------------------------>/nandb
[partition]
    name         = rootfs
    size         = 753664
    downloadfile = "rootfs.fex"
    verify       = 1

There was no 3rd partition, maybe it leads to my issue. So I decided to add a third partition to fill my disk.

I have a 512MB flash, here is my formula to process the correct size:
16384 KiB => 16,777,216 B (MBR size)
32768*512B => 16,777,216 B (/dev/nanda partition)
753664*512 => 385,875,968 B (/dev/nandb partition)
512MB - 16.7MB*2 - 385.8MB = 92,569,600

I don't know why, but my packer tool complains when I'm not using a multiple of 16,7MB for the partition size.

So the closest size is: 83,886,080 => 163,840 sector of 512 B

So my new sys_partition.fex is:

Code: [Select]
[mbr]
size = 16384

[partition_start]

;------------------------------>/nanda
[partition]
    name         = bootloader
    size         = 32768
    downloadfile = "bootloader.fex"
    verify       = 1

;------------------------------>/nandb
[partition]
    name         = rootfs
    size         = 753664
    downloadfile = "rootfs.fex"
    verify       = 1

;------------------------------>/nandc
[partition]
    name         = swap
    size         = 163840
    verify       = 1

; 16960 * 512 bytes not used (partitions must be multiple of 16777216 B)

And after burning this image, I execute nand_part and cat /proc/partitions, here is the output:

Code: [Select]
check partition table copy 0: mbr: version 0x00000200, magic softw411
OK
check partition table copy 1: mbr: version 0x00000200, magic softw411
OK
check partition table copy 2: mbr: version 0x00000200, magic softw411
OK
check partition table copy 3: mbr: version 0x00000200, magic softw411
OK
mbr: version 0x00000200, magic softw411
4 partitions
partition  1: class =         DISK, name =   bootloader, partition start =    32768, partition size =    32768 user_type=0
partition  2: class =         DISK, name =       rootfs, partition start =    65536, partition size =   753664 user_type=0
partition  3: class =         DISK, name =         swap, partition start =   819200, partition size =   163840 user_type=0
partition  4: class =         DISK, name =        UDISK, partition start =   983040, partition size =   -92160 user_type=0




major minor  #blocks  name

  93        0     498688 nand
  93        1      16384 nanda
  93        2     376832 nandb
  93        3      81920 nandc
  93        4       7168 nandd

From nand-part output, UDISK seems to be auto generated but now it has a negative size, -92160 but with my understanding it should be 16960 because of my previous calculation.

From /cat/partitions, the nandd size seems not correct too but different though:
7168 KiB => 7,340,032 B => 14,336 sector of 512B and not 16960 as it should be

I must admit that I'm lost. Maybe I made a mistake in my calculation. Let me know if something is more clear for you.

Thanks for your help,

Arthur
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 28, 2016, 07:50:34 pm
Hi Arthur,

I've been looking at your posts here again.  I'd drop the idea of adding your swap partition because it might cause more trouble than it's worth.

The original problem was the incorrect size of the UDISK partition which is apparently added by Livesuit.  The numbers in your original /proc/partitions list do add up if you include the 32768 sectors preceding nanda.  But the total (498688) times 1024 is awfully close to 512MB.  My 8GB chip here is marked as 7700MB which is a lot less than 8GB.  I doubt it's bad blocks because the total is the same on the five CTs I've checked.

If you manually partition your NAND do you get the same mismatch ?

Your comment about partitions and offsets being a multiple of 16.7MiB might be to allow for a NAND block size of 16.7MiB.  The risk I see in having partitions starting mid-way through a block is that all NAND writes are done in blocks and it seems much safer to never affect a partition when updating an adjacent partition.  I've seen "alignment error" messages from u-boot which is probably due to me having partitions on 128kiB boundaries.  I did this because that's the block size in the chip here.  I'll try setting the granularity to 16.7MiB and see if that stops the warnings.

Are you trying to create a Livesuit image for distribution ?  If not can you try manually partitioning the NAND ?

Cheers,
Steven

Here is some info from my CT here:

Code: [Select]
root@home:~# cat /proc/partitions
major minor  #blocks  name

  93        0    7520256 nand
  93        1      65536 nand1
  93        2    7372800 nand2
  93        3      65536 nand3
   8        0  488386584 sda
   8        1   41943040 sda1
   8        2   41943040 sda2
   8        3  209715200 sda3
   8        4  194784280 sda4
root@home:~# nand-part -f a20
check partition table copy 0: mbr: version 0x00000200, magic softw411
OK
check partition table copy 1: mbr: version 0x00000200, magic softw411
OK
check partition table copy 2: mbr: version 0x00000200, magic softw411
OK
check partition table copy 3: mbr: version 0x00000200, magic softw411
OK
mbr: version 0x00000200, magic softw411
3 partitions
partition  1: class =         DISK, name =         boot, partition start =    32768, partition size =   131072 user_type=0
partition  2: class =         DISK, name =       rootfs, partition start =   163840, partition size = 14745600 user_type=0
partition  3: class =         DISK, name =        dummy, partition start = 14909440, partition size =   131072 user_type=0
root@home:~# fdisk -l

Disk /dev/nand: 7700 MB, 7700742144 bytes
255 heads, 63 sectors/track, 936 cylinders, total 15040512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Title: Re: A10/A20 NAND boot block access
Post by: phelum on May 05, 2016, 03:24:34 am
Hi Arthur,

I've just loaded the NAND on an Olimex A20 board using my software.  I investigated the alignment error problem you mentioned (start/size is not align) and found that all partitions must be aligned to super-blocks rather than just NAND blocks.  A super-block is all the blocks (generally 2) on a plane in the NAND chip.

So I changed my calcs to use super-blocks and now I don't get any warning when the system is booting.

But the size mismatch error on the last partition still occurs.  I checked the partitioning done by my software and everything is correct.  If I run nand-part this also shows the correct start keys and sizes.  But lsblk (uses proc/partitions) is still wrong.  In my case here a 4MiB partition is reported as 28MiB.  Same goes if I mount it.  For safety I'd leave this last partition unformatted and unused.

Code: [Select]
root@lime:/media# nand-part -f a20
check partition table copy 0: mbr: version 0x00000200, magic softw411
OK
check partition table copy 1: mbr: version 0x00000200, magic softw411
OK
check partition table copy 2: mbr: version 0x00000200, magic softw411
OK
check partition table copy 3: mbr: version 0x00000200, magic softw411
OK
mbr: version 0x00000200, magic softw411
4 partitions
partition  1: class =         DISK, name =   bootloader, partition start =     8192, partition size =     8192 user_type=0
partition  2: class =         DISK, name =         boot, partition start =    16384, partition size =     8192 user_type=0
partition  3: class =         DISK, name =      CloudFS, partition start =    24576, partition size =  7700480 user_type=0
partition  4: class =         DISK, name =        UDISK, partition start =  7725056, partition size =     8192 user_type=0
root@lime:/media# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
nand         93:0    0   3.7G  0 disk
??nand1      93:1    0     4M  0 part
??nand2      93:2    0     4M  0 part
??nand3      93:3    0   3.7G  0 part
??nand4      93:4    0    28M  0 part /media
mmcblk0     179:0    0   7.4G  0 disk
??mmcblk0p1 179:1    0   7.4G  0 part /
root@lime:/media#

Update: I've just found what looks like the culprit here.
block/partitions/sunxi_nand.c:
Code: [Select]
static void sunxi_nand_parse_mbr(struct parsed_partitions *state, int no)
{
Sector sect;
int part_cnt;
struct MBR *mbr = 0;
char b[BDEVNAME_SIZE];

bdevname(state->bdev, b);

mbr = read_part_sector(state, mbr_sector, &sect);
for (part_cnt = 0; part_cnt < mbr->tag.PartCount &&
/* the sunxi mbr structure ver 0x200 allows for 120
* partitions but only 31 fit into a page so forget the
* ones that do not fit. */
part_cnt < MAX_PART_COUNT && part_cnt < 31;
part_cnt++) {
/* special case: last partition uses up rest of NAND space */
__u32 size = mbr->array[part_cnt].lenlo;
if (part_cnt == mbr->tag.PartCount - 1)
size = get_capacity(state->bdev->bd_disk) -
mbr->array[part_cnt].addrlo;
printk(KERN_WARNING "Dev Sunxi %s %s: part %d, start %d, size %d\n",
MBR_MAGIC, b, part_cnt + 1,
mbr->array[part_cnt].addrlo, size);
put_partition(state, part_cnt + 1,
mbr->array[part_cnt].addrlo, size);
}
strlcat(state->pp_buf, "\n", PAGE_SIZE);
put_dev_sector(sect);
}

Notice the test in the middle that changes the size of the last partition.  There might be a good reason but it doesn't seem nice to me.

After patching the kernel to remove the special handling for the last partition I get:
Code: [Select]
root@lime:~# lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
nand         93:0    0   3.7G  0 disk
??nand1      93:1    0     4M  0 part
??nand2      93:2    0     4M  0 part
??nand3      93:3    0   3.7G  0 part
??nand4      93:4    0     4M  0 part
mmcblk0     179:0    0   7.4G  0 disk
??mmcblk0p1 179:1    0   7.4G  0 part /
root@lime:~#

The 4M for nand4 is correct.  So the size mismatch is just a bug in block/partitions/sunxi_nand.c.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: arthur on May 06, 2016, 05:57:09 am
Dear Steven,

Thank you very much for your analysis !

I'll patch the driver right now to avoid this mismatch and, like you said, I'll avoid to use this last partition for safety.

Have a nice day  ;)

Arthur
Title: Re: A10/A20 NAND boot block access
Post by: phelum on May 06, 2016, 06:19:56 am
I'll patch the driver right now to avoid this mismatch and, like you said, I'll avoid to use this last partition for safety.
Hi Arthur,

If you patch the driver it should be safe to format and use the last partition.  Maybe just make sure the start key and size values are correct in case there's another bug/feature.

I've seen mention that if the last partition in the nand-part specs is size 0 then it ends up taking the rest of the NAND.  It looks like the driver ALWAYS makes the last partition extend to the end of the NAND.  But unfortunately it gets the size and therefore the end wrong.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on March 24, 2018, 05:44:16 am
Hello,

Does this:

"linux-sunxi u-boot is fully SPL enabled which means it supports booting directly on the bare metal with no help from the Allwinner bootloaders. U-Boot SPL fully replaces Allwinner boot0 & boot1."

from here:

https://github.com/linux-sunxi/u-boot-sunxi/wiki

mean, u-boot can be flashed directly on nand and Allwinner boot0 and boo1 can be forgotten?
Title: Re: A10/A20 NAND boot block access
Post by: phelum on March 24, 2018, 06:13:29 am
Hello,

Does this:

"linux-sunxi u-boot is fully SPL enabled which means it supports booting directly on the bare metal with no help from the Allwinner bootloaders. U-Boot SPL fully replaces Allwinner boot0 & boot1."

from here:

https://github.com/linux-sunxi/u-boot-sunxi/wiki

mean, u-boot can be flashed directly on nand and Allwinner boot0 and boo1 can be forgotten?
Hi,

I think the U-Boot with SPL you've found is for SD cards rather than NAND.  I suspect the boot0/boot1 portion (written to block 8 onwards) is totally different to that in a NAND setup.  I never managed to compile boot0/boot1 but I did extract it and change the DRAM specs to suit a board with memory chips that aren't defined in the standard tables.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on March 24, 2018, 07:44:06 am
I think the U-Boot with SPL you've found is for SD cards rather than NAND.

I would like to update u-boot in NAND. So - do you think I should use one without SPL? Should it be flashed to the very beginning of the flash if I do not have drivers for boot/boot1 access installed?

I would like to have only u-boot and environment on NAND and everything else on SATA.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on March 24, 2018, 08:20:01 am
I would like to update u-boot in NAND. So - do you think I should use one without SPL? Should it be flashed to the very beginning of the flash if I do not have drivers for boot/boot1 access installed?

I would like to have only u-boot and environment on NAND and everything else on SATA.
Hi,

I think you'll need at least a small VFAT partition in NAND just to give you somewhere to put U-Boot.  boot1 finds U-Boot and runs it and this loads the kernel (e.g. uImage) which by default is in this partition.  It might be possible to put the kernel on SDA and change uEnv.txt to specify this if U-Boot is smart enough to access a volume (probably ext4) on disk.

If you just want to update U-Boot then you'll find it in /dev/mmcblk0p1 and it's called u-boot.bin.  The entry in uEnv.txt to specify a different kernel is kernel=........ but I don't know how you'd specify a kernel on sda1 (something like /dev/sda1/boot/mykernel but that's not right).  Maybe ask on the Armbian forum if you get no answers here.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on March 24, 2018, 10:43:04 am
Hi,

I think you'll need at least a small VFAT partition in NAND just to give you somewhere to put U-Boot.  boot1 finds U-Boot and runs it and this loads the kernel (e.g. uImage) which by default is in this partition.  It might be possible to put the kernel on SDA and change uEnv.txt to specify this if U-Boot is smart enough to access a volume (probably ext4) on disk.

If you just want to update U-Boot then you'll find it in /dev/mmcblk0p1 and it's called u-boot.bin.  The entry in uEnv.txt to specify a different kernel is kernel=........ but I don't know how you'd specify a kernel on sda1 (something like /dev/sda1/boot/mykernel but that's not right).  Maybe ask on the Armbian forum if you get no answers here.

Cheers,
Steven

2017.11 U-Boot seems to identify HDD as expected.

Would it be just possible to load kernel and what ever more needed from HDD with ext4load command?

Best solution would be if U-Boot environment was on NAND and U-Boot would load kernel and everything from HDD if it was present. SD-card would be used if it was found (before HDD in boot order) and booting everything from NAND if there were not HDD and not SD-card.

I suppose this can be done if the U-Boot is new enough. This 2017.11 at least seems to support both HDD and SD-card. I'm not sure if it can boot from NAND without compiling it again with NAND-support.

This U-Boot was from an Armbian image.

B.T.W. This suggests there should be U-Boot with SPL also put on NAND:

https://lists.denx.de/pipermail/u-boot/2015-May/215159.html

Might this just depend on if there is an U-Boot supporting both SATA and NAND?
Title: Re: A10/A20 NAND boot block access
Post by: phelum on March 25, 2018, 12:00:20 am
2017.11 U-Boot seems to identify HDD as expected.

Would it be just possible to load kernel and what ever more needed from HDD with ext4load command?

Best solution would be if U-Boot environment was on NAND and U-Boot would load kernel and everything from HDD if it was present. SD-card would be used if it was found (before HDD in boot order) and booting everything from NAND if there were not HDD and not SD-card.

I suppose this can be done if the U-Boot is new enough. This 2017.11 at least seems to support both HDD and SD-card. I'm not sure if it can boot from NAND without compiling it again with NAND-support.

This U-Boot was from an Armbian image.

B.T.W. This suggests there should be U-Boot with SPL also put on NAND:

https://lists.denx.de/pipermail/u-boot/2015-May/215159.html

Might this just depend on if there is an U-Boot supporting both SATA and NAND?

Hi,
Sorry for the delayed reply.  I can't login using Firefox and I only have Chrome at one worksite.

It sounds like you've researched U-Boot well and know more about it than I do.  U-Boot does have an internal script (that can be changed) which tells it where to look for the kernel and in what order.  I remember that Igor changed a U-Boot so it looked for uEnv.cb2 if running on a CB2 and uEnv.ct if running on a CT.  So lots can be done to U-Boot if you want to change it.

I don't think your statement about U-Boot with SPL is quite right for NAND.  With NAND the SPL equivalent is boot0 and boot1 and these are hidden away in part of the NAND that is very hard to access.  The modified disk driver I offered is patched so it can read/write this area but only before normal access is done.  NAND has complications such as relocation and usage tables and also there is a scrambling key which affects reads and writes to the boot area.

If your CT is booting correctly I'd forget about boot0 and boot1 and just play with the relevant files in /dev/nand1.  The relevant files there are u-boot.bin and uEnv.txt if you've got an old non-dts setup.  Newer setups use boot.scr and a .dtb file.  The boot.scr file specifies the load sequence so you should be able to change this file and get the CT to load the kernel from SDA (or whatever).  The only boot.scr I have was for a CB2 loading a 3.16 kernel from MMC.  If your U-Boot can access NAND or disk then you should be able to change boot.scr and get U-Boot to load a kernel from disk or NAND. 

Here is my boot.scr (without header line prepended by mkimage):
Code: [Select]
env set fdt_high ffffffff
ext2load mmc 0:1 0x44000000 /boot/uImage
#ext2load mmc 0:1 0x46000000 /boot/sun7i-a20-cubieboard2.dtb
ext2load mmc 0:1 0x46000000 /boot/cb2.dtb
#ext2load mmc 0:1 0x46000000 /boot/sun4i-a10-cubieboard.dtb
setenv bootargs console=ttyS0,115200 root=/dev/mmcblk0p3 rootwait panic=10 rootfstype=ext2
setenv extraargs rootfstype=ext2
#setenv machid 0x10bb
#setenv bootm_boot_mode sec  << this forces single CPU mode !!!
bootm 0x44000000 - 0x46000000


It loads the kernel at 44000000, the .dtb at 46000000, and then boots the kernel (at 44000000) which gets the .dtb address in memory from the last parameter.  So the kernel starts and knows where to find the .dtb (environment) file.  mmc 0:1 is SD card drive 0 partition 1 but I don't know what you'd put for NAND or disk here.

Does this help ? 

-- Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 02, 2018, 04:04:13 pm
Thank you for your knowledge.

What I and far more my son has done now, is compiling and installing 03.2018 version of U-Boot in it. It really replaces Allwinner boot0 and boot1. It seems to boot U-boot well and reliably.

It seems to be good choice in booting from NAND to SATA without SD-card. I'm sure it would as well boot from USB and of course SD-card too.

I also would like it to boot (for emergency situations only) from NAND, but this seems to be tricky to do. U-boot and kernel both can access NAND, but they seem to do it somewhat differently. Both use mainline drivers.

Do you know if it has something to do with ECC or some NAND-related bit-manipulation?
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 03, 2018, 03:39:45 am
I also would like it to boot (for emergency situations only) from NAND, but this seems to be tricky to do. U-boot and kernel both can access NAND, but they seem to do it somewhat differently. Both use mainline drivers.

Do you know if it has something to do with ECC or some NAND-related bit-manipulation?
Hi,

When you say your new U-boot works well I assume you're booting from SD-card.  Is this correct ?

My understanding of a NAND boot sequence is boot0, boot1, boot.axf. u-boot.bin, uImage.
boot0 initialises the RAM and loads an runs boot1.
boot1 reads the first NAND partition (vfat) and loads and runs boot.axf.
boot.axf reads boot.ini and loads and runs u-boot.bin.
u-boot.bin reads uEnv.txt and loads and starts the kernel (uImage).

I think u-boot.bin searches for a boot script in /boot or / and will use it if found.  Otherwise it defaults to its internal script which tells it to load script.bin and then load and run the kernel.

I presume you have a Cubietruck.  Can you connect a serial monitor to the debug port and see what happens if you try to boot with no SD-card present ?  My CB1, CB2, and CTs all boot from NAND but I'm running a legacy 3.4.103 kernel.  I did have the CB2 booting a 3.16 kernel but I gave up because it wouldn't support something (I forget what).

If your CT won't boot from NAND then watching the log from the debug port is the first step in solving the issues.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 03, 2018, 03:08:52 pm
I also would like it to boot (for emergency situations only) from NAND, but this seems to be tricky to do. U-boot and kernel both can access NAND, but they seem to do it somewhat differently. Both use mainline drivers.

Do you know if it has something to do with ECC or some NAND-related bit-manipulation?
Hi,

When you say your new U-boot works well I assume you're booting from SD-card.  Is this correct ?

No. It does boot 03/2018 U-Boot from NAND without SD-card.

See this:  http://git.denx.de/?p=u-boot.git;a=blob_plain;f=board/sunxi/README.nand;hb=HEAD

All you have to change is not flashing u-boot-dtb.bin but u-boot-dtb.img instead.

Do you know if kernel 4.9 should be compatible with this U-Boot version? It very much sounds like U-Boot and this kernel version use NAND different ways and they do not work together. Everything else seems to be OK.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 03, 2018, 04:17:12 pm
Hi,

When you say your new U-boot works well I assume you're booting from SD-card.  Is this correct ?

No. It does boot 03/2018 U-Boot from NAND without SD-card.

See this:  http://git.denx.de/?p=u-boot.git;a=blob_plain;f=board/sunxi/README.nand;hb=HEAD

All you have to change is not flashing u-boot-dtb.bin but u-boot-dtb.img instead.

Do you know if kernel 4.9 should be compatible with this U-Boot version? It very much sounds like U-Boot and this kernel version use NAND different ways and they do not work together. Everything else seems to be OK.
Hi,
Wow, I'm impressed with how things have changed and what you've done.  What output do you get when you try to boot from NAND ?  I'm guessing that U-boot starts and tries to start the kernel.  One problem that I used to hit was that the MACHID wasn't passed to the kernel correctly and when the kernel started it noticed the mismatch and just stopped with no error message.

Also, where did you get your 4.9 kernel from ?  I might have to download a copy and try it here.

-- Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 04, 2018, 06:42:14 am
Hi,
Wow, I'm impressed with how things have changed and what you've done.  What output do you get when you try to boot from NAND ?  I'm guessing that U-boot starts and tries to start the kernel.  One problem that I used to hit was that the MACHID wasn't passed to the kernel correctly and when the kernel started it noticed the mismatch and just stopped with no error message.

Also, where did you get your 4.9 kernel from ?  I might have to download a copy and try it here.

What we or mostly my son has done so far, is booting U-Boot from NAND without any USB-device or SD-card attached and then putting SD-card or USB-stick in loading kernel from there. Then booting kernel.

We did try flash NAND with user space having kernel and all then, but there are still some issues. So in fact we are still in situaton where U-Boot can definitely be booted from NAND and it works just fine with user space in USB or SD-card. We have not tried SATA yet, but I expect it work anyways and only the NAND is an issue.

NAND can be erased. It can be flashed, but after booting again it contains carbage. There has to be something odd. Writing NAND (through kernel MTD drivers) and then reading same NAND area using U-Boot driver to load user space is the issue. Kernel writing and U-Boot reading do not match.

Kernel source was cloned from bbrezillion gihub linux-sunxi having branch bb/4.7/ubi-mlc. It is 4.9 version even though branch is 4.7.

https://github.com/bbrezillon/linux-sunxi/tree/bb/4.7/ubi-mlc

I suppose there must be something changed and old kernel NAND writing does not match new U-Boot reading. If we only got them to match each other, this would work just fine.

There are two attachments now. If you take 3/2018 mainline U-boot, patch it with what you can find in one of the attachments and then use u-boot.config as .config, you can have the very same U-Boot we have here.

Then you can take kernel source using link above, patch it with a file in attached kernel patch attachment and use config from the same zip, your kernel will be there too.

Just don't forget to use U-Boot ".img" file instead of ".bin" to flash. Mainline flashing instruction is otherwise just fine.
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 04, 2018, 07:47:58 am
The u-boot patch seems to have one setting what was not really tried.

Its the last line having "CONFIG_NAND_ECC_NONE=y" in it in Cubietruck_defconfig.

I suppose it should be removed first if you are going to try this.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 04, 2018, 08:19:51 am
The u-boot patch seems to have one setting what was not really tried.

Its the last line having "CONFIG_NAND_ECC_NONE=y" in it in Cubietruck_defconfig.

I suppose it should be removed first if you are going to try this.
Things seem to have changed.  I downloaded U-Boot and it's now 2018.05.  The Cubietruck_defconfig doesn't have any mention of NAND.  If I run make menuconfig I can set the NAND support options but it needs the page, block, and OOB sizes which I'll have to get.  Also the make complains that my gcc isn't new enough so I might have to try compiling on a newer system.  This will take a while.

With your U-Boot and kernel writes problem (garbage) I remember I had to use different read/writes in my modified NAND driver to access the boot0/boot1 area.  It looks like U-Boot now lives in this area rather than the normal NAND blocks so maybe the reads/writes it uses aren't right for normal NAND (the area you can partition using nand-part) and this is why you're getting garbage with normal I/O from the kernel.

I presume you've partitioned and formatted your NAND.  Does it have a small (say 64MiB) first partition formated VFAT ?  If so can you mount it and see if u-boot.bin is there ?

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 04, 2018, 08:31:52 am
Things seem to have changed.  I downloaded U-Boot and it's now 2018.05.  The Cubietruck_defconfig doesn't have any mention of NAND.  If I run make menuconfig I can set the NAND support options but it needs the page, block, and OOB sizes which I'll have to get.

If you patch it, you will have NAND chip metrics in Cubietruck_defconfig.

With your U-Boot and kernel writes problem (garbage) I remember I had to use different read/writes in my modified NAND driver to access the boot0/boot1 area.  It looks like U-Boot now lives in this area rather than the normal NAND blocks so maybe the reads/writes it uses aren't right for normal NAND (the area you can partition using nand-part) and this is why you're getting garbage with normal I/O from the kernel.

I presume you've partitioned and formatted your NAND.  Does it have a small (say 64MiB) first partition formated VFAT ?  If so can you mount it and see if u-boot.bin is there ?

We did not get far enough to make partitions. U-Boot SPL and U-Boot itself are read from bare NAND without any partitioning. U-Boot settings have one partition leaving start of the NAND untouched for those and U-Boot environment.

The U-boot can use NAND right, because environment can be saved and again read without any trouble.

To get the very same mainline U-Boot source it might be good idea to use this link:

ftp://ftp.denx.de/pub/u-boot/u-boot-2018.03.tar.bz2
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 04, 2018, 08:51:02 am
To get the very same mainline U-Boot source it might be good idea to use this link:

ftp://ftp.denx.de/pub/u-boot/u-boot-2018.03.tar.bz2
Thanks for the U-Boot link.  It sounds like new U-Boot does live in the low area of NAND.  Would it be acceptable to you if I get the new kernel running with an old U-Boot ?  I'll download Boris's kernel tomorrow and try to make one for testing.

-- Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 04, 2018, 09:18:22 am
Thanks for the U-Boot link.  It sounds like new U-Boot does live in the low area of NAND.  Would it be acceptable to you if I get the new kernel running with an old U-Boot ?  I'll download Boris's kernel tomorrow and try to make one for testing.

You are welcome to do it.

Having recent SW running on it is only important in keeping it up to date.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 05, 2018, 04:11:48 am
Having recent SW running on it is only important in keeping it up to date.
Hi,

I'm getting the 4.7 kernel running (from SD card) but I can't get it to see the AP6210 or NAND.  What I have noticed is that there is the CONFIG_MTD_NAND_ECC_SMC option in the compiler config that reverses the work byte order.  If you're seeing garbage maybe changing this option might help.

-- Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 09, 2018, 05:43:19 am
This project is now in a situation, where U-Boot 2018.03 sits on NAND and everything else is on SATA-disk.

U-Boot loads and executes kernel, ramdisk and dtb from SATA, but kernel boot does hung like this:

*********************************************

resetting ...

U-Boot SPL 2018.03 (Apr 08 2018 - 00:01:35 +0300)
DRAM: 2048 MiB
CPU: 912000000Hz, AXI/AHB/APB: 3/2/2
Trying to boot from NAND


U-Boot 2018.03 (Apr 08 2018 - 00:01:35 +0300) Allwinner Technology

CPU:   Allwinner A20 (SUN7I)
Model: Cubietech Cubietruck
I2C:   ready
DRAM:  2 GiB
NAND:  8192 MiB
MMC:   SUNXI SD/MMC: 0
Loading Environment from NAND... OK
Setting up a 1024x768 vga console (overscan 0x0)
In:    serial
Out:   vga
Err:   vga
Allwinner mUSB OTG (Peripheral)
SCSI:  Target spinup took 0 ms.
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst
Net:   eth0: ethernet@01c50000, eth1: usb_ether
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
USB2:   USB EHCI 1.00
USB3:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 2 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0
scanning bus for devices...
  Device 0: (0:0) Vendor: ATA Prod.: ST9320325AS Rev: 0001
            Type: Hard Disk
            Capacity: 305245.3 MB = 298.0 GB (625142448 x 512)
Found 1 device(s).

Device 0: (0:0) Vendor: ATA Prod.: ST9320325AS Rev: 0001
            Type: Hard Disk
            Capacity: 305245.3 MB = 298.0 GB (625142448 x 512)
... is now current device
Scanning scsi 0:1...
Found U-Boot script /boot/boot.scr
3708 bytes read in 49 ms (73.2 KiB/s)
## Executing script at 43100000
Boot script loaded from scsi
170 bytes read in 23 ms (6.8 KiB/s)
5824915 bytes read in 200 ms (27.8 MiB/s)
6909240 bytes read in 214 ms (30.8 MiB/s)
Found legacy kernel configuration
** File not found /boot/script.bin **
## Loading init Ramdisk from Legacy Image at 43300000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    5824851 Bytes = 5.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Loading Ramdisk to 49a71000, end 49fff153 ... OK

Starting kernel ...

*****************************************

Any suggestions what to do next to get it booting the whole way up?
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 09, 2018, 06:08:59 am
This project is now in a situation, where U-Boot 2018.03 sits on NAND and everything else is on SATA-disk.

U-Boot loads and executes kernel, ramdisk and dtb from SATA, but kernel boot does hung like this:
Found U-Boot script /boot/boot.scr
3708 bytes read in 49 ms (73.2 KiB/s)
## Executing script at 43100000
Boot script loaded from scsi
170 bytes read in 23 ms (6.8 KiB/s)
5824915 bytes read in 200 ms (27.8 MiB/s)
6909240 bytes read in 214 ms (30.8 MiB/s)
Found legacy kernel configuration
** File not found /boot/script.bin **
## Loading init Ramdisk from Legacy Image at 43300000 ...
   Image Name:   uInitrd
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    5824851 Bytes = 5.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Loading Ramdisk to 49a71000, end 49fff153 ... OK

Starting kernel ...

*****************************************

Any suggestions what to do next to get it booting the whole way up?

In your /boot/boot.cmd file find your "setenv bootargs" statement and add "machid=0x10bb" to the end.  Then use mkimage to create a new /boot/boot.scr.

I've tried compiling U-Boot 2018.03 to add NAND code but I keep getting syntax errors or sram size exceeds available area.  So now I'm trying to get an old U-Boot with fdt capability to support NAND.

The 4.7 kernel you're using doesn't seem to support NAND.  I've tried changing the config but I just get it it to detect NAND.

So my progress so far has been minimal.  I'll post here when I get some useful results.

-- Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 09, 2018, 10:24:09 am
Any suggestions what to do next to get it booting the whole way up?

OK. Needed second eyes again to make it work. My son has so much younger eyes ;)

I did not have .next file in /boot folder. It did try to boot old kernel. This prevented it coming up.

Now I have U-Boot and U-Boot environment in NAND, but everything else in SATA-disk. It boots without MMC-card as I wanted it to be.

Still there is not a way to use NAND as boot device for the whole system, but this should be possible by just flashing kernel in raw mode in NAND and loading it with UBI drivers from there. Then the rest of the userspace could be put in UBI.


B.T.W I did put SPL and U-boot in first erase block and environment in second just to be able to erase them separately.

The above attachment having U-Boot config has at least one error. Nand setting block size has to be added one more zero. Even with this corrected I did not manage it to work. There has to be something weird in scrambling bits.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 12, 2018, 04:24:49 pm
Still there is not a way to use NAND as boot device for the whole system, but this should be possible by just flashing kernel in raw mode in NAND and loading it with UBI drivers from there. Then the rest of the userspace could be put in UBI.
I've been through the config options in the 4.7 kernel you mentioned but I find any way to get it to support NAND.  With your CT does 'lsblk' show any NAND partitions ?  I'm thinking it's unlikely that any kernel can be booted from NAND if it can't access the NAND.  For instance, how could it load modules ?

I've installed an Armbian kernel with stretch on a CT and can generate U-Boot.  I'm still sarching for config settings so it can access NAND.  I'd like an SD-card version that can load my old kernels in NAND.

Please check 'lsblk' in your 4.7 kernel and tell me what it detects.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 20, 2018, 06:17:17 am
Please check 'lsblk' in your 4.7 kernel and tell me what it detects.

Hi,

I don't have it installed any more, but maybe you should use the same version I did. It's here:

https://github.com/bbrezillon/linux-sunxi/tree/bb/4.7/ubi-mlc

It seems to be vestion 4.9 even though name would suggest 4.7.

In my present installation kernel version is 4.14.7-sunxi. Only U-Boot is on NAND. Kernel having initramfs and modules are all loaded from SATA. My goal was to get rid of SD-card and this does it.

However kernel version 4.9 did show NAND but the present 4.14.7-sunxi does not.

If you want the exact source code I used to make this U-Boot and 4.9 kernel, just give me some email address to send them. I have FTP server here, but it would be extremely slow.
Title: Re: A10/A20 NAND boot block access
Post by: phelum on April 20, 2018, 07:02:01 am
In my present installation kernel version is 4.14.7-sunxi. Only U-Boot is on NAND. Kernel having initramfs and modules are all loaded from SATA. My goal was to get rid of SD-card and this does it.

However kernel version 4.9 did show NAND but the present 4.14.7-sunxi does not.
Thanks for the info.  What you've got now sounds like a new version of the setup I use which is U-Boot on NAND and everything else on SATA.  I went this way because they say NAND fails after many writes and so I figured the best thing to do is just keep static boot stuff there and get everything else from disk.

I haven't bothered updating from my old Wheezy system because it works well and supports NAND and the AP6210 (Wi-Fi).  The new Armbian kernel (4.18) supports the AP6210 but not NAND.  I can't get it to load with an old U-Boot.  But the Armbian release (stretch) will compile the new U-Boot which is good.

I've been away from all this for over a year and things certainly have changed.  People seem very critical of NAND and I guess it's not as good as was hoped.  So I'd say you've probably got the best setup at the moment.

Cheers,
Steven
Title: Re: A10/A20 NAND boot block access
Post by: tilator on April 20, 2018, 08:11:51 am
So I'd say you've probably got the best setup at the moment.

Best is a bit strong word, but it definitely fulfills my needs right now.