12.4.4.4 Wide-character versions

Support for exchange of wide-character strings is still under consideration. The functions dealing with 8-bit character strings return failure when operating on a wide-character atom or Prolog string object. The functions below can extract and unify both 8-bit and wide atoms and string objects. Wide character strings are represented as C arrays of objects of the type pl_wchar_t, which is guaranteed to be the same as wchar_t on platforms supporting this type. For example, on MS-Windows, this represents a 16-bit UTF-16 string, while using the GNU C library (glibc) this represents 32-bit UCS4 characters.

atom_t PL_new_atom_wchars(size_t len, const pl_wchar_t *s)

Create atom from wide-character string as PL_new_atom_nchars() does for ISO-Latin-1 strings. If s only contains ISO-Latin-1 characters a normal byte-array atom is created. If len is (size_t)-1, it is computed from s using wcslen(). See PL_new_atom() for error handling.

const pl_wchar_t* PL_atom_wchars(atom_t atom, size_t *len)

Extract characters from a wide-character atom. Succeeds on any atom marked as ‘text’. If the underlying atom is a wide-character atom, the returned pointer is a pointer into the atom structure. If the atom is represented as an ISO-Latin-1 string, the returned pointer comes from Prolog's ‘buffer stack’ (see section 12.4.14).

bool PL_get_wchars(term_t t, size_t *len, pl_wchar_t **s, unsigned flags)

Wide-character version of PL_get_chars(). The flags argument is the same as for PL_get_chars(). Note that this operation may return a pointer into Prolog's ‘buffer stack’ (see section 12.4.14).

bool PL_put_wchars(term_t -t, int type, size_t len, const pl_wchar_t *s)

Put text from a wide character array in t. Arguments are the same as PL_unify_wchars().^{221The current implementation uses PL_put_variable() followed by PL_unify_wchars().}

bool PL_unify_wchars(term_t +t, int type, size_t len, const pl_wchar_t *s)

Unify t with a textual representation of the C wide-character array s. The type argument defines the Prolog representation and is one of PL_ATOM, PL_STRING, PL_CODE_LIST or PL_CHAR_LIST.

bool PL_unify_wchars_diff(term_t +t, term_t -tail, int type, size_t len, const pl_wchar_t *s)

Difference list version of PL_unify_wchars(), only supporting the types PL_CODE_LIST and PL_CHAR_LIST. It serves two purposes. It allows for returning very long lists from data read from a stream without the need for a resizing buffer in C. Also, the use of difference lists is often practical for further processing in Prolog. Examples can be found in packages/clib/readutil.c from the source distribution.

int PL_wcwidth(int chr)

Display column width of a Unicode code point chr. Returns 0 for combining and other zero-width characters, 2 for East Asian Wide and Fullwidth characters, 1 for ordinary printable characters, and -1 for control characters and code points unknown to the implementation.

The result is locale-independent and identical on every supported platform. Width data is sourced at table-build time from EastAsianWidth.txt (UAX #11) and the general-category property, and stored as two bits in each uflags_map entry alongside the syntax classification used by the parser; runtime lookups are a single byte fetch. The Unicode version that drives the table is reported by the unicode_syntax_version flag (see section 2.15.1.9).

The argument is a full 32-bit code point. Avoid casting to wchar_t before the call: wchar_t is 16-bit on Windows, and the cast silently drops non-BMP characters.

int PL_is_id_start(int chr)

int PL_is_id_continue(int chr)

int PL_is_uppercase(int chr)

int PL_is_decimal(int chr)

int PL_is_layout(int chr)

Unicode classification predicates. Each returns non-zero iff the 32-bit code point chr carries the corresponding property as defined by the data files in src/Unicode/ and consulted by the Prolog reader. Provided so foreign extensions and embedded toolkits (notably xpce) classify code points exactly as SWI-Prolog does, locale- independently. id_start and id_continue follow Prolog's identifier syntax (close to UAX #31 XID_Start / XID_Continue, adjusted for Prolog). layout matches the reader's whitespace set (white_space/1 in src/Unicode/derived_core_properties.pl).

unsigned int PL_ctype_flags(int chr)

Returns the POSIX character classes of the 32-bit code point chr as the bitwise or of the constants below, e.g.

if ( PL_ctype_flags(c) & PL_CTYPE_ALPHA )
  ...

PL_CTYPE_ALNUM	PL_CTYPE_ALPHA or a numeric character
PL_CTYPE_ALPHA	Unicode Alphabetic
PL_CTYPE_CNTRL	general category Cc or Cf
PL_CTYPE_DIGIT	0 ...9, as POSIX demands
PL_CTYPE_GRAPH	printable and not white space
PL_CTYPE_LOWER	Unicode Lowercase
PL_CTYPE_PRINT	PL_CTYPE_GRAPH or white space
PL_CTYPE_PUNCT	printable, neither alnum nor space
PL_CTYPE_SPACE	Unicode White_Space
PL_CTYPE_UPPER	Unicode Uppercase

This is the classifier behind code_type/2 and char_type/2 and a locale-independent replacement for the <wctype.h> functions iswalpha(), iswspace(), ... Besides not varying with LC_CTYPE, it does not vary between C libraries (glibc and Darwin disagree on whether U+00A0 and U+0085 are white space, for example) and it answers for the full Unicode range also on Windows, where wint_t is 16 bits and the <wctype.h> functions silently fail for non-BMP code points.

int PL_toupper(int chr)

int PL_tolower(int chr)

int PL_totitle(int chr)

Unicode simple case conversion of the 32-bit code point chr, replacing towupper(), towlower() and towctrans(). Code points that have no mapping are returned unchanged.

Simple means one code point in, one code point out, so conversion is length preserving: U+00DF LATIN SMALL LETTER SHARP S and the ligatures are returned unchanged rather than expanded to SS, FI, ... as SpecialCasing.txt would have it.

As with the classes, the mapping comes from the Unicode Character Database and is therefore the same in every locale --- notably it does not apply the Turkish and Azeri dotted/dotless i rules --- and on every platform, and it works for non-BMP code points also on Windows.