rockbox/utils/nwztools/database/nvp
Igor Skochinsky 03dd4b92be nwztools/database: misc improvements
* make gen_db.py work on Windows/Python 2

- use hashlib module instead of md5sum, also don't rely on / for file path
matching
- don't use 'file' for a variable name

* fix parse_nvp_header.sh for older kernels

pre-emmc kernel sources use a slightly different #define format; adjust
regexp to catch it.

* add nwz-x1000 series NVP layout (from icx1087_nvp.h)

some new tags have no description, alas the driver doesn't have
them :/

*  minor fixes to nvp/README

fixed typos/wording

Change-Id: I77d8c2704be2f2316e32aadcfd362df7102360d4
2017-04-25 11:24:24 +10:00
..
Makefile nwztools: small cleanups 2017-04-25 11:21:54 +10:00
nodes-nw-a20.txt nwztools: rename nwz-a20 to nw-a20, that was a typo 2017-01-08 12:09:49 +01:00
nodes-nw-zx100.txt nwztools: fix typo (nwz-zx100 -> nw-zx100) 2017-01-09 21:48:43 +01:00
nodes-nwz-a10.txt
nodes-nwz-a820.txt
nodes-nwz-a860.txt
nodes-nwz-e450.txt
nodes-nwz-e460.txt
nodes-nwz-e580.txt
nvptool.cpp nwztools: small cleanups 2017-04-25 11:21:54 +10:00
nw-a20.txt nwztools: rename nwz-a20 to nw-a20, that was a typo 2017-01-08 12:09:49 +01:00
nw-a30.txt
nw-e060.txt
nw-e080.txt nwztools/database: add database of information on Sony NWZ linux players 2016-11-11 16:07:14 +01:00
nw-s640.txt nwztools: add various info about S740, S750, S640, E050 2017-01-08 22:34:57 +01:00
nw-s780.txt
nw-wm1.txt nwztools: add NW-WM1 nvp table, regenerate database 2017-01-04 17:03:54 +01:00
nw-zx100.txt nwztools: fix typo (nwz-zx100 -> nw-zx100) 2017-01-09 21:48:43 +01:00
nwz-a10.txt
nwz-a840.txt nwztools: add various info about S740, S750, S640, E050 2017-01-08 22:34:57 +01:00
nwz-a850.txt
nwz-a860.txt nwztools/database: add database of information on Sony NWZ linux players 2016-11-11 16:07:14 +01:00
nwz-e050.txt nwztools: add various info about S740, S750, S640, E050 2017-01-08 22:34:57 +01:00
nwz-e350.txt
nwz-e450.txt
nwz-e460.txt
nwz-e470.txt
nwz-e580.txt
nwz-s740.txt nwztools: add various info about S740, S750, S640, E050 2017-01-08 22:34:57 +01:00
nwz-s750.txt nwztools: add various info about S740, S750, S640, E050 2017-01-08 22:34:57 +01:00
nwz-s760.txt
nwz-x1000.txt nwztools/database: misc improvements 2017-04-25 11:24:24 +10:00
parse_all_nvp_headers.sh
parse_all_nvp_nodes.sh
parse_nvp_header.sh nwztools/database: misc improvements 2017-04-25 11:24:24 +10:00
parse_nvp_nodes.sh nwztools/database: misc improvements 2017-04-25 11:24:24 +10:00
README nwztools/database: misc improvements 2017-04-25 11:24:24 +10:00

The NVP map varies a lot from players to players, it is inconceivable to build
it by hand. The approach taken is to extract it from the kernel of each player.
Since Sony provides the kernel of all players, it is 'only' a matter of
downloading all of them. A bit of background on the NVP: it is non-volatile 
partition of the flash that is divided in regions and then "zones". 
Each "zone" stores the data of a "node". The ABI
between the NVP driver and the userspace is an index: the userspace gives the
index of a node, and then drives looks up its table to see where it is and what
is its size. The index map changes over time so Sony introduces standard "names"
for its entries, those are 3-letters acronym (for example "fup" or "bti" or "shp")
that have a corresponding index. Sometimes the driver also contains a
description of the nodes, in English (e.g. "bti" stands for "boot image").

parse_nvp_header.sh
===================

This script takes a header filename, a kernel directory or a kernel tgz and will
try to extract the mapping automatically. It produces a list of pairs
  <node>,<name>
where <node> is the index of the node (that's the only thing that is usable on
a running device) and <name> is the standard name of the node. The output should
be written to <series name>.txt
Note that <name> is an acronym (like 'fup') and the description needs to be generated
separately (see nvprool section).

parse_all_nvp_headers.sh
========================

This script expects a directory to have the following structure:
  dir/
    nwz-a10/
      linux-kernel-*.tgz
    nwz-e460/
      linux-kernel-*.tgz
    ...
Each sudirectory must be the series name (as used in ../series.txt) and the kernel
must be a tgz (end in .tgz and not .tar.gz) of the form linux-kernel-*.tgz. Usually
the variable bit will be the version but some kernels have unknown versions. It
will then run parse_nvp_header.sh on each of them and store the result in a
file called <series name>.txt

NOTE: the kernel can be symlinks to other files

nvptool
=======

The kernel headers do not contain descriptions of the nvp node names.
They can be extracted from the icx_nvp[_emmc].ko driver on target using complicated
elf parsing done by nvptool. Technically nvptool can discover much more information
like the node -> human name mapping as well and the actual zone in the flash but
since we can already extract it easily from the headers, we only extract description
names from it.

parse_all_nvp_nodes.sh
======================

This script expects a directory to have the following structure:
  dir/
    nwz-a10/
      rootfs.tgz
    nwz-e460/
      rootfs.tgz
    ...
Each sudirectory must be the series name (as used in ../series.txt) and the rootfs
must be a tar. It will then extract the relevant icx_nvp driver from it and run
nvptool on it to produce a file called nodes-<series name>.txt

NOTE: the rootfs can be symlinks to other files