Many includes of fat.h are pointless. Some includes are just for
SECTOR_SIZE. Add a file 'firmware/include/fs_defines.h' for that
and to define tuneable values that were scattered amongst various
headers.
Remove some local definitions of SECTOR_SIZE since they have to be
in agreement with the rest of the fs code anyway.
(We'll see what's in fact pointless in a moment ;)
Change-Id: I9ba183bf58bd87f5c45eba7bd675c7e2c1c18ed5
* 8 bits is enough to allow 260 character base names when five
bytes is the minimum indirect storage size (0..255->5..260).
* Don't truncate anything that's too long as that can lead to
bad behavior, simply don't include the offending entry in the
parent.
* Set the .tinyname flag to 1 by default to indicate that
the entry's name doesn't need freeing. Clear it only when
allocating indirect storage.
* Rename some things to help catch all instances
Change-Id: Iff747b624acbb8e03ed26c24afdf0fc715fd9d99
These flags aren't stored for an open file because they're simply
actions for open() to take, corresponding to O_CREAT and O_EXCL.
Just pass the oflag argument along to the deeper call, with some
minor filtering.
Change-Id: Ic8bcfba718ebf4228bdc45de3088af1974820557
The file system rework introduced incompatibility between dircache
and the tagcache ramcache and playlist dircache path caching. This
update makes changes to filesystem code to reintegrate all that.
It also fixes a couple bugs that were found when vetting all the
code. The filestream cache was being reset without regard to
the stream even if it was shared in write mode (made work of
.playlist_control). Better handling of unmounting gives files a
better go at force-closing them without risk to disk integrity.
Did some miscellaneous pedantic changes. Improved efficiency of
testing a file's existence (a little) since the path parser will
be shared between file code and parsing for the sake of finding
dircache references, not duplicated as before.
This commit doesn't reenable said items just for the sake of
keeping changes separate and related.
Plan for the next is to enable dircache again for the playlists
(easy peasy) and reenable tagcache ramcache but *without* the
dircache path caching because it's rather substantial to change
in itself. The ramcache will still function without dircache.
Change-Id: I7e2a9910b866251fa8333e1275f72fcfc8425d2d
Somehow it got hooked to simulator file functions when it should
be (and was) using raw OS functions.
Credit: Frank Gevaerts
Change-Id: Iac02fed1067830a432183632a047e00dfd03d3c2
Document the fact that buffers are movable by default.
Care must be taken to not pass them to functions that yield().
Also clarify other things:
- Passing NULL as "ops" to buflib_alloc_ex() causes
buffers to be movable by default (but not shrinkable).
- If you want shrinkable buffers during compaction,
you have to provide a shrink callback.
- To disable buffer movement, you have to pass NULL
for the move_callback inside the callback structure.
- The concept of default callbacks was removed
long ago, remove the only reference of it.
Change-Id: I3bf0ea6b08b507d80a19f3c2c835aca32b3f7800
Turns out maemo's old gcc 4.2.1 doesn't include any arch
optimized swapXX() functions, just plain C implementations.
Before we pull in lots of linux kernel headers for the
C implementation, just stick to rockbox's own version.
Change-Id: Ic28b41b52fe47f814c7f3897ce15334a42b6c5e2
A GNU extension that returns dst + size instead of dst. It's a nice
shortcut when copying strings with a known size or back-to-back blocks
and you have to do it often.
May of course be called directly or alternately through
__builtin_mempcpy in some compiler versions.
For ASM on native targets, it is implemented as an alternate entrypoint
to memcpy which adds minimal code and overhead.
Change-Id: I4cbb3483f6df3c1007247fe0a95fd7078737462b
This patch redoes the filesystem code from the FAT driver up to the
clipboard code in onplay.c.
Not every aspect of this is finished therefore it is still "WIP". I
don't wish to do too much at once (haha!). What is left to do is get
dircache back in the sim and find an implementation for the dircache
indicies in the tagcache and playlist code or do something else that
has the same benefit. Leaving these out for now does not make anything
unusable. All the basics are done.
Phone app code should probably get vetted (and app path handling
just plain rewritten as environment expansions); the SDL app and
Android run well.
Main things addressed:
1) Thread safety: There is none right now in the trunk code. Most of
what currently works is luck when multiple threads are involved or
multiple descriptors to the same file are open.
2) POSIX compliance: Many of the functions behave nothing like their
counterparts on a host system. This leads to inconsistent code or very
different behavior from native to hosted. One huge offender was
rename(). Going point by point would fill a book.
3) Actual running RAM usage: Many targets will use less RAM and less
stack space (some more RAM because I upped the number of cache buffers
for large memory). There's very little memory lying fallow in rarely-used
areas (see 'Key core changes' below). Also, all targets may open the same
number of directory streams whereas before those with less than 8MB RAM
were limited to 8, not 12 implying those targets will save slightly
less.
4) Performance: The test_disk plugin shows markedly improved performance,
particularly in the area of (uncached) directory scanning, due partly to
more optimal directory reading and to a better sector cache algorithm.
Uncached times tend to be better while there is a bit of a slowdown in
dircache due to it being a bit heavier of an implementation. It's not
noticeable by a human as far as I can say.
Key core changes:
1) Files and directories share core code and data structures.
2) The filesystem code knows which descriptors refer to same file.
This ensures that changes from one stream are appropriately reflected
in every open descriptor for that file (fileobj_mgr.c).
3) File and directory cache buffers are borrowed from the main sector
cache. This means that when they are not in use by a file, they are not
wasted, but used for the cache. Most of the time, only a few of them
are needed. It also means that adding more file and directory handles
is less expensive. All one must do in ensure a large enough cache to
borrow from.
4) Relative path components are supported and the namespace is unified.
It does not support full relative paths to an implied current directory;
what is does support is use of "." and "..". Adding the former would
not be very difficult. The namespace is unified in the sense that
volumes may be specified several times along with relative parts, e.g.:
"/<0>/foo/../../<1>/bar" :<=> "/<1>/bar".
5) Stack usage is down due to sharing of data, static allocation and
less duplication of strings on the stack. This requires more
serialization than I would like but since the number of threads is
limited to a low number, the tradoff in favor of the stack seems
reasonable.
6) Separates and heirarchicalizes (sic) the SIM and APP filesystem
code. SIM path and volume handling is just like the target. Some
aspects of the APP file code get more straightforward (e.g. no path
hashing is needed).
Dircache:
Deserves its own section. Dircache is new but pays homage to the old.
The old one was not compatible and so it, since it got redone, does
all the stuff it always should have done such as:
1) It may be update and used at any time during the build process.
No longer has one to wait for it to finish building to do basic file
management (create, remove, rename, etc.).
2) It does not need to be either fully scanned or completely disabled;
it can be incomplete (i.e. overfilled, missing paths), still be
of benefit and be correct.
3) Handles mounting and dismounting of individual volumes which means
a full rebuild is not needed just because you pop a new SD card in the
slot. Now, because it reuses its freed entry data, may rebuild only
that volume.
4) Much more fundamental to the file code. When it is built, it is
the keeper of the master file list whether enabled or not ("disabled"
is just a state of the cache). Its must always to ready to be started
and bind all streams opened prior to being enabled.
5) Maintains any short filenames in OEM format which means that it does
not need to be rebuilt when changing the default codepage.
Miscellaneous Compatibility:
1) Update any other code that would otherwise not work such as the
hotswap mounting code in various card drivers.
2) File management: Clipboard needed updating because of the behavioral
changes. Still needs a little more work on some finer points.
3) Remove now-obsolete functionality such as the mutex's "no preempt"
flag (which was only for the prior FAT driver).
4) struct dirinfo uses time_t rather than raw FAT directory entry
time fields. I plan to follow up on genericizing everything there
(i.e. no FAT attributes).
5) unicode.c needed some redoing so that the file code does not try
try to load codepages during a scan, which is actually a problem with
the current code. The default codepage, if any is required, is now
kept in RAM separarately (bufalloced) from codepages specified to
iso_decode() (which must not be bufalloced because the conversion
may be done by playback threads).
Brings with it some additional reusable core code:
1) Revised file functions: Reusable code that does things such as
safe path concatenation and parsing without buffer limitations or
data duplication. Variants that copy or alter the input path may be
based off these.
To do:
1) Put dircache functionality back in the sim. Treating it internally
as a different kind of file system seems the best approach at this
time.
2) Restore use of dircache indexes in the playlist and database or
something effectively the same. Since the cache doesn't have to be
complete in order to be used, not getting a hit on the cache doesn't
unambiguously say if the path exists or not.
Change-Id: Ia30f3082a136253e3a0eae0784e3091d138915c8
Reviewed-on: http://gerrit.rockbox.org/566
Reviewed-by: Michael Sevakis <jethead71@rockbox.org>
Tested: Michael Sevakis <jethead71@rockbox.org>
After a local commit, any file that included version.h would have to
be recompiled on account of the changed version string. This changes
version.h in the build directory to rbversion.h and includes the
preprocessor macro from rbversion.h in firmware/common/version.c so
that only that one file needs to be recompiled after a local commit
rather than a whole slew of them.
Change-Id: I900d97e3a24a0610698283416d97b4fa3a3a2cf6
Reviewed-on: http://gerrit.rockbox.org/937
Reviewed-by: Michael Sevakis <jethead71@rockbox.org>
Tested: Michael Sevakis <jethead71@rockbox.org>
We redefine the top-level macros to our own in order to maintain
compatibility with compound initializers by wrapping the mid or low
level definitions from the OS header.
This allows, hopefully optimized, macros from the host OS's headers to
be used when building any hosted target obviating the need for
NEED_GENERIC_BYTESWAPS unless the target simply doesn't define its
own optimized versions (MIPS!).
Throw in some 64-bit swaps for completeness' sake; they generate no code
if not yet used anyway.
Change-Id: I21b384b55fea46833d01ea3cad1ad8952ea01a11
Forms implemented to a greater or lesser degree at the moment:
ll_* = singly-linked list
lld_* = doubly-linked list
lldc_* = doubly-linked circular list
Change-Id: Ieed5af50fc59165c8b14c3513b3b5d0e6f7de9fa
Any number of readers may be in the critical section at a time and writers
are mutually exclusive to all other threads. They are a better choice when
data is rarely modified but often read and multiple threads can safely
access it for reading.
Priority inheritance is fully implemented along with other changes to the
kernel to fully support it on multiowner objects.
This also cleans up priority code in the kernel and updates some associated
structures in existing objects to the cleaner form.
Currently doesn't add the mrsw_lock.[ch] files since they're not yet
needed by anything but the supporting improvements are still useful.
This includes a typed bitarray API (bitarray.h) which is pretty basic
for now.
Change-Id: Idbe43dcd9170358e06d48d00f1c69728ff45b0e3
Reviewed-on: http://gerrit.rockbox.org/801
Reviewed-by: Michael Sevakis <jethead71@rockbox.org>
Tested: Michael Sevakis <jethead71@rockbox.org>
This doesn't touch external tools as I see no need for.
Change-Id: Ia69248c4b6a033c3772916525257e3540bddcffa
Reviewed-on: http://gerrit.rockbox.org/891
Tested: Sebastian Leonhardt <sebastian.leonhardt@web.de>
Reviewed-by: Marcin Bukat <marcin.bukat@gmail.com>
A thread polls the appropriate GPIO pin for sd card presence and mounts
using the mount system call.
Change-Id: I31ab41c4120f4af64eb6998b7e7b6f9051585efb
Part of this change is to align sdlapp builds to other application targets
in that the sim_* wrappers are not used anymore (except for sim_read/write).
Path mangling is now done in rbpaths.c as well.
Change-Id: I9726da73b50a83d9e1a1840288de16ec01ea029d
The iocharset mount option names the codepages slightly differently and
must be translated properly.
Change-Id: I147a256e3453136282244201c27225a30cdfdca0
This is the basic port to the new target Samsung
YP-R1, which runs on a similar platform as YP-R0.
Port is usable, although there are still
some optimizations that have to be done.
Change-Id: If83a8e386369e413581753780c159026d9e41f04
This allows buflib clients to more accurately estimate the total memory usage.
It's still not 100% accurate because the handle table grows in blocks, thus
buflib might use more memory that caused by allocations directly.
Change-Id: I68338bb94f510ad188fcb588aebf895b5f9197c5
This should catch the case of buffer misuse which results
in corrupted cookie of next allocation. The check is performed
on move_block() so it may be a bit late.
There is buflib_check_valid() provided which checks the
integrity of all cookies for given context.
On DEBUG build with --sdl-thread this check is carried out
for core_ctx on every context switch to catch problems earlier.
Change-Id: I999d4576084592394e3dbd3bdf0f32935ff5f601
Reviewed-on: http://gerrit.rockbox.org/711
Reviewed-by: Thomas Martitz <kugel@rockbox.org>
The font engine can now perform cache lookups even if the font file is
closed, if the font was disabled with the new font_disable_all() function.
It is highly probable that the lookup succeeds but in the cache-miss case
a blank, full-width glyph will be returned.
Change-Id: I3c97e747d2a0ba30c7b495c6118c9f029d265b56
This function relocates a buflib back buffer, updating pointers in struct
buflib_context. It does not move any data by itself.
The intended use-case is buflib-on-buflib, where a buflib back buffer is
allocated with buflib and attempted to be moved. The move_callback() can call
this and return BUFLIB_CB_OK on success. No move_callback() is called for the
subordinate buflib buffer, therefore it must not contain non-movable
allocations. The caller is generally responsible moving the data and all its
implications.
Change-Id: I869219f9cff786a172c9e917a5f34470073892e6
This function will now ask shrinkable allocations to give up all of their
memory. With future support of playback.c this can be used as a safe
replacement for audio_get_buffer().
Change-Id: I290a51d2c75254e66baf5698c41dc444dea6247a
buflib_allocatable() is what buflib_available() was before (it was in fact
simply renamed). It returns the largest contiguous block of memory. This
can be allocated and will definitely succeed, although larger allocations
may also succeed if the buffer can be compacted and shrinked.
buflib_available() now counts all free bytes, contiguous or not. This
better matches the description and how the caller use it.
Change-Id: I511e4eb5f4cf1821d957b3f4ef8a685ce40fe289
Reviewed-on: http://gerrit.rockbox.org/481
Reviewed-by: Thomas Martitz <kugel@rockbox.org>
Tested-by: Thomas Martitz <kugel@rockbox.org>
The idea is to share loading code between bootloaders and rolo().
Change-Id: I1656ed91946d7a05cb7c9fa7a16793c3c862a5cd
Reviewed-on: http://gerrit.rockbox.org/190
Reviewed-by: Marcin Bukat <marcin.bukat@gmail.com>
This is going right in since it's long overdue. If anything is goofed,
drop me a line or just tweak it yourself if you know what's wrong. :-)
Make HW/SW codec interface more uniform when emulating HW functionality
on SWCODEC for functions such as "audiohw_set_pitch". The firmware-to-
DSP plumbing is in firmware/drivers/audiohw-swcodec.c. "sound_XXX"
APIs are all in sound.c with none in DSP code any longer.
Reduce number of settings definitions needed by each codec by providing
defaults for common ones like balance, channels and SW tone controls.
Remove need for separate SIM code and tables and add virtual codec header
for hosted targets.
Change-Id: I3f23702bca054fc9bda40f49824ce681bb7f777b
Will need it soon enough.
Combine the contents of all the various fixedpoint.h files.
Not moving fixedpoint.c for now since I'm not sure where it
should be and it causes some dependency issues.
Change-Id: Ideacbca2ca78f9158c2b114b113c274f68e908d5
It was also broken functionally, probably since a while.So restore the
functionality. Run it on the dap, the tcd files will be placed into .rockbox folder.
Change-Id: Id7a6ce4389dfaf99799258902be80d630af0601c
In Europe Windows defaults to its own codepage cp1252 (also known as "WinLatin"
or "Windows-1252"). cp1252 adds some characters to ISO-8859-1.
Some mp3 tagging software on Windows uses cp1252 instead of ISO-8859-1. This
violates the ID3 specification, which requires tags to be ISO-8859-1 or
Unicode. However, similar violations are made for other codepages and supported
by Rockbox using the "Default Codepage" setting. Add support for cp1252 to
enable people using such broken tools to override the correct decoding to get
their tags displayed properly.
Change-Id: I9f2ec478afe2503e99ee8e6609416c92b0f453e0
Reviewed-on: http://gerrit.rockbox.org/209
Reviewed-by: Jens Arnold <amiconn@rockbox.org>
Tested-by: Jens Arnold <amiconn@rockbox.org>
This changes the way creat() is wrapped around in native builds
so more experienced devs should look at it.
This patch forces to compile fat test in 32bit mode. Building
natively on x86-64 works just fine but our fat code apparently
can't deal with 64bit pointers/ints correctly.
Change-Id: I000015094f7db957ce826c22672608cd419908b0
Reviewed-on: http://gerrit.rockbox.org/228
Reviewed-by: Thomas Martitz <kugel@rockbox.org>
Buffers are not allocated and thread is not created until the first
call where voice is required.
Adds a different callback (sync_callback) to buflib so that other
sorts of synchonization are possible, such as briefly locking-out the
PCM callback for a buffer move. It's sort of a messy addition but it
is needed so voice decoding won't have to be stopped when its buffer
is moved.
Change-Id: I4d4d8c35eed5dd15fb7ee7df9323af3d036e92b3
* filesize() is not POSIX, so it doesn't need stubbing or redirecting
* make the various directory functions use the sim_ versions for PCTOOL
* PCTOOL needs generic byteswap functions
* fix the database makefile to not use -DSIMULATOR anymore
Change-Id: Ic6abc4f662830b85626c751a472fa4a03e844871
This port is a hybrid native/RaaA port. It runs on a embedded linux system,
but is the only application. It therefore can implement lots of stuff that
native targets also implement, while leveraging the underlying linux kernel.
The port is quite advanced. User interface, audio playback, plugins work
mostly fine. Missing is e.g. power mangement and USB (see SamsungYPR0 wiki page).
Included in utils/ypr0tools are scripts and programs required to generate
a patched firmware. The patched firmware has the rootfs modified to load
Rockbox. It includes a early/safe USB mode.
This port needs a new toolchain, one that includes glibc headers and libraries.
rockboxdev.sh can generate it, but e.g. codesourcey and distro packages may
also work.
Most of the initial effort is done by Lorenzo Miori and others (on ABI),
including reverse engineering and patching of the original firmware,
initial drivers, and more. Big thanks to you.
Flyspray: FS#12348
Author: Lorenzo Miori, myself
Merry christmas to ypr0 owners! :)
git-svn-id: svn://svn.rockbox.org/rockbox/trunk@31415 a1c6a512-1295-4272-9138-f99709370657
