nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
The NVP map varies a lot from players to players, it is inconceivable to build
|
|
|
|
|
it by hand. The approach taken is to extract it from the kernel of each player.
|
|
|
|
|
Since Sony provides the kernel of all players, it is 'only' a matter of
|
2017-04-03 13:13:46 +00:00
|
|
|
downloading all of them. A bit of background on the NVP: it is non-volatile
|
|
|
|
|
partition of the flash that is divided in regions and then "zones".
|
|
|
|
|
Each "zone" stores the data of a "node". The ABI
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
between the NVP driver and the userspace is an index: the userspace gives the
|
2017-04-03 13:13:46 +00:00
|
|
|
index of a node, and then drives looks up its table to see where it is and what
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
is its size. The index map changes over time so Sony introduces standard "names"
|
|
|
|
|
for its entries, those are 3-letters acronym (for example "fup" or "bti" or "shp")
|
2017-04-03 13:13:46 +00:00
|
|
|
that have a corresponding index. Sometimes the driver also contains a
|
|
|
|
|
description of the nodes, in English (e.g. "bti" stands for "boot image").
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
|
|
|
|
|
parse_nvp_header.sh
|
|
|
|
|
===================
|
|
|
|
|
|
2017-04-03 13:13:46 +00:00
|
|
|
This script takes a header filename, a kernel directory or a kernel tgz and will
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
try to extract the mapping automatically. It produces a list of pairs
|
|
|
|
|
<node>,<name>
|
|
|
|
|
where <node> is the index of the node (that's the only thing that is usable on
|
2017-04-03 13:13:46 +00:00
|
|
|
a running device) and <name> is the standard name of the node. The output should
|
|
|
|
|
be written to <series name>.txt
|
|
|
|
|
Note that <name> is an acronym (like 'fup') and the description needs to be generated
|
|
|
|
|
separately (see nvprool section).
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
|
|
|
|
|
parse_all_nvp_headers.sh
|
|
|
|
|
========================
|
|
|
|
|
|
2017-04-03 13:13:46 +00:00
|
|
|
This script expects a directory to have the following structure:
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
dir/
|
|
|
|
|
nwz-a10/
|
|
|
|
|
linux-kernel-*.tgz
|
|
|
|
|
nwz-e460/
|
2017-04-03 13:13:46 +00:00
|
|
|
linux-kernel-*.tgz
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
...
|
2017-04-03 13:13:46 +00:00
|
|
|
Each sudirectory must be the series name (as used in ../series.txt) and the kernel
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
must be a tgz (end in .tgz and not .tar.gz) of the form linux-kernel-*.tgz. Usually
|
|
|
|
|
the variable bit will be the version but some kernels have unknown versions. It
|
|
|
|
|
will then run parse_nvp_header.sh on each of them and store the result in a
|
|
|
|
|
file called <series name>.txt
|
|
|
|
|
|
|
|
|
|
NOTE: the kernel can be symlinks to other files
|
|
|
|
|
|
|
|
|
|
nvptool
|
|
|
|
|
=======
|
|
|
|
|
|
2017-04-03 13:13:46 +00:00
|
|
|
The kernel headers do not contain descriptions of the nvp node names.
|
|
|
|
|
They can be extracted from the icx_nvp[_emmc].ko driver on target using complicated
|
|
|
|
|
elf parsing done by nvptool. Technically nvptool can discover much more information
|
|
|
|
|
like the node -> human name mapping as well and the actual zone in the flash but
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
since we can already extract it easily from the headers, we only extract description
|
|
|
|
|
names from it.
|
|
|
|
|
|
|
|
|
|
parse_all_nvp_nodes.sh
|
|
|
|
|
======================
|
|
|
|
|
|
2017-04-03 13:13:46 +00:00
|
|
|
This script expects a directory to have the following structure:
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
dir/
|
|
|
|
|
nwz-a10/
|
|
|
|
|
rootfs.tgz
|
|
|
|
|
nwz-e460/
|
|
|
|
|
rootfs.tgz
|
|
|
|
|
...
|
2017-04-03 13:13:46 +00:00
|
|
|
Each sudirectory must be the series name (as used in ../series.txt) and the rootfs
|
nwztools/database: add database of information on Sony NWZ linux players
There must be an evil genius in Sony's Walkman division. Someone who made sure
that each model is close enough to the previous one so that little code is needed
but different enough so that an educated guess is not enough.
Each linux-based Sony player has a model ID (mid) which is a 32-bit integer.
I was able to extract a list of all model IDs and the correspoding name of
the player (see README). This gives us 1) a nice list of all players (because
NWZ-A729 vs NWZ-A729B, really Sony?) 2) an easy way to find the name of player
programatically. It seems that the lower 8-bit of the model ID gives the storage
size but don't bet your life on it. The remaining bytes seem to follow some kind
of pattern but there are exceptions.
From this list, I was able to build a list of all Sony's series (up to quite
recent one). The only safe way to build that is by hand, with a list of series,
each series having a list of model IDs. The notion of series is very important
because all models in a series share the same firmware.
A very important concept on Sony's players is the NVP, an area of the flash
that stores data associated with keys. The README contains more information but
basically this is where is record the model ID, the destination, the boot flags,
the firmware upgrade flags, the boot image, the DRM keys, and a lot of other stuff.
Of course Sony decided to slightly tweak the index of the keys regularly over time
which means that each series has a potentially different map, and we need this map
to talk to the NVP driver. Fortunately, Sony distributes the kernel for all its
players and they contain a kernel header with this information. I wrote a script
to unpack kernel sources and parse this header, producing a bunch of nw-*.txt
files, included in this commit. This map is very specific though: it maps Sony's
3-letter names (bti) to indexes (1). This is not very useful without the
decription (bti = boot image) and its size (262144). This information is harder
to come by, and is only stored in one place: if icx_nvp_emmc.ko drivers, found
on the device. Fortunately, Sony distributes a number of firmware upgrade, that
contain the rootfs, than once extracted contain this driver. The driver is a
standard ELF files with symbols. I wrote a parsing tool (nvptool) that is able
to extract this information from the drivers. Using that, I produced a bunch
of nodes-nw*.txt files. A reasonable assumption is that nodes meaning and
size do not change over time (bti is always the boot image and is always
262144 bytes), so by merging a few of those file, we can get a complete picture
(note that some nodes that existed in older player do not exists anymore so
we really need to merge several ones from different generations).
The advantage of storing all this information in plain text files, is that it
now makes it easy to parse it and produce whatever format we want to use it.
I wrote a python script that parses all this mess and produces a C file and
header with all this information (nwz_db.{c,h}).
Change-Id: Id790581ddd527d64418fe9e4e4df8e0546117b80
2016-11-11 14:40:56 +00:00
|
|
|
must be a tar. It will then extract the relevant icx_nvp driver from it and run
|
|
|
|
|
nvptool on it to produce a file called nodes-<series name>.txt
|
|
|
|
|
|
|
|
|
|
NOTE: the rootfs can be symlinks to other files
|