Previously the CJK character detection defined only characters in the
range U+4000..U+AFFF as "CJK characters". This excludes an incredibly
large number of CJK characters within the BMP, let alone the whole two
planes dedicated to rarer CJK characters (the SIP and TIP). As a result,
a very large number of Chinese, Japanese, and Korean characters were not
detected as being CJK characters.
While slightly less elegant-looking, it is far more accurate to compute
the codepoint from the utf8 character and then see if it falls within
one of the defined CJK blocks. This is not future-proof against future
CJK ideograph extensions in future Unicode versions, but there is no
real way to accurately predict such changes so this is the best we can
do without accidentally treating characters explicitily defined as being
non-CJK in Unicode as CJK.
While we're at it, copy Lua 5.3's utf8.charpattern constant definition
so that we can more easily write utf8 iterators with string.gmatch (at
least in the interim until there is a rework of utf8 handling in
KOReader and everything is rebuilt on top of utf8proc).
Some unit tests are added for Korean and Japanese text, and the existing
unit tests needed a minor adjustment to handle the fact that
isSplittable now correctly detects CJK punctuation as a character to
compare against the forbidden split rules.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
* Use paintRect and plain colors instead of lightenRect and a weird
dimming factor.
* Update call sites to the new API
* Handle FP maths properly (i.e., floor coordinates & ceil dimensions at
the latest possible time).
* Fix border handling in the fill bar (make sure we actually honor it
when computin the x position, and that we won't overflow into it when
computing the width).
* Update docs
Reword "Purge .sdr" to "Reset settings".
When purging, remove only the known document metadata
files, and not those for a document with the same name but
a different suffix.
Bookmarks list:
- page numbers are displayed
- page bookmarks are marked with a star
- new setting: Sort by largest page number (default: checked)
New bookmark setting: Add page number / timestamp to bookmark
- If enabled (default), bookmark name is 'Page # notes @ time'.
- If disabled, bookmark name is equal to the notes field.
Rename bookmark dialog:
- page number and timestamp are displayed in the input
dialog description
- blank input renames bookmark to the default name in
accordance with the new setting
Also fix: changing boundaries of the highlight: the name of the
highlight is not changed if it was previously edited by the user.
Aleksa Sarai