unorm2.h File Reference

C API: New API for Unicode Normalization. More...

#include "unicode/utypes.h"
#include "unicode/localpointer.h"
#include "unicode/uset.h"

Go to the source code of this file.

Typedefs

typedef struct UNormalizer2 UNormalizer2
 C typedef for struct UNormalizer2.

Enumerations

enum  UNormalization2Mode { UNORM2_COMPOSE, UNORM2_DECOMPOSE, UNORM2_FCD, UNORM2_COMPOSE_CONTIGUOUS }
 

Constants for normalization modes.

More...
enum  UNormalizationCheckResult { UNORM_NO, UNORM_YES, UNORM_MAYBE }
 

Result values for normalization quick check functions.

More...

Functions

const UNormalizer2unorm2_getInstance (const char *packageName, const char *name, UNormalization2Mode mode, UErrorCode *pErrorCode)
 Returns a UNormalizer2 instance which uses the specified data file (packageName/name similar to ucnv_openPackage() and ures_open()/ResourceBundle) and which composes or decomposes text according to the specified mode.
UNormalizer2unorm2_openFiltered (const UNormalizer2 *norm2, const USet *filterSet, UErrorCode *pErrorCode)
 Constructs a filtered normalizer wrapping any UNormalizer2 instance and a filter set.
void unorm2_close (UNormalizer2 *norm2)
 Closes a UNormalizer2 instance from unorm2_openFiltered().
int32_t unorm2_normalize (const UNormalizer2 *norm2, const UChar *src, int32_t length, UChar *dest, int32_t capacity, UErrorCode *pErrorCode)
 Writes the normalized form of the source string to the destination string (replacing its contents) and returns the length of the destination string.
int32_t unorm2_normalizeSecondAndAppend (const UNormalizer2 *norm2, UChar *first, int32_t firstLength, int32_t firstCapacity, const UChar *second, int32_t secondLength, UErrorCode *pErrorCode)
 Appends the normalized form of the second string to the first string (merging them at the boundary) and returns the length of the first string.
int32_t unorm2_append (const UNormalizer2 *norm2, UChar *first, int32_t firstLength, int32_t firstCapacity, const UChar *second, int32_t secondLength, UErrorCode *pErrorCode)
 Appends the second string to the first string (merging them at the boundary) and returns the length of the first string.
UBool unorm2_isNormalized (const UNormalizer2 *norm2, const UChar *s, int32_t length, UErrorCode *pErrorCode)
 Tests if the string is normalized.
UNormalizationCheckResult unorm2_quickCheck (const UNormalizer2 *norm2, const UChar *s, int32_t length, UErrorCode *pErrorCode)
 Tests if the string is normalized.
int32_t unorm2_spanQuickCheckYes (const UNormalizer2 *norm2, const UChar *s, int32_t length, UErrorCode *pErrorCode)
 Returns the end of the normalized substring of the input string.
UBool unorm2_hasBoundaryBefore (const UNormalizer2 *norm2, UChar32 c)
 Tests if the character always has a normalization boundary before it, regardless of context.
UBool unorm2_hasBoundaryAfter (const UNormalizer2 *norm2, UChar32 c)
 Tests if the character always has a normalization boundary after it, regardless of context.
UBool unorm2_isInert (const UNormalizer2 *norm2, UChar32 c)
 Tests if the character is normalization-inert.

Detailed Description

C API: New API for Unicode Normalization.

Unicode normalization functionality for standard Unicode normalization or for using custom mapping tables. All instances of UNormalizer2 are unmodifiable/immutable. Instances returned by unorm2_getInstance() are singletons that must not be deleted by the caller. For more details see the Normalizer2 C++ class.

Definition in file unorm2.h.


Typedef Documentation

typedef struct UNormalizer2 UNormalizer2

C typedef for struct UNormalizer2.

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4

Definition at line 117 of file unorm2.h.


Enumeration Type Documentation

Constants for normalization modes.

For details about standard Unicode normalization forms and about the algorithms which are also used with custom mapping tables see http://www.unicode.org/unicode/reports/tr15/

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
Enumerator:
UNORM2_COMPOSE 

Decomposition followed by composition.

Same as standard NFC when using an "nfc" instance. Same as standard NFKC when using an "nfkc" instance. For details about standard Unicode normalization forms see http://www.unicode.org/unicode/reports/tr15/

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UNORM2_DECOMPOSE 

Map, and reorder canonically.

Same as standard NFD when using an "nfc" instance. Same as standard NFKD when using an "nfkc" instance. For details about standard Unicode normalization forms see http://www.unicode.org/unicode/reports/tr15/

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UNORM2_FCD 

"Fast C or D" form.

If a string is in this form, then further decomposition without reordering would yield the same form as DECOMPOSE. Text in "Fast C or D" form can be processed efficiently with data tables that are "canonically closed", that is, that provide equivalent data for equivalent text, without having to be fully normalized. Not a standard Unicode normalization form. Not a unique form: Different FCD strings can be canonically equivalent. For details see http://www.unicode.org/notes/tn5/#FCD

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UNORM2_COMPOSE_CONTIGUOUS 

Compose only contiguously.

Also known as "FCC" or "Fast C Contiguous". The result will often but not always be in NFC. The result will conform to FCD which is useful for processing. Not a standard Unicode normalization form. For details see http://www.unicode.org/notes/tn5/#FCC

Draft:
This API may be changed in the future versions and was introduced in ICU 4.4

Definition at line 42 of file unorm2.h.

Result values for normalization quick check functions.

For details see http://www.unicode.org/reports/tr15/#Detecting_Normalization_Forms

Stable:
ICU 2.0
Enumerator:
UNORM_NO 

The input string is not in the normalization form.

Stable:
ICU 2.0
UNORM_YES 

The input string is in the normalization form.

Stable:
ICU 2.0
UNORM_MAYBE 

The input string may or may not be in the normalization form.

This value is only returned for composition forms like NFC and FCC, when a backward-combining character is found for which the surrounding text would have to be analyzed further.

Stable:
ICU 2.0

Definition at line 91 of file unorm2.h.


Function Documentation

int32_t unorm2_append ( const UNormalizer2 norm2,
UChar first,
int32_t  firstLength,
int32_t  firstCapacity,
const UChar second,
int32_t  secondLength,
UErrorCode pErrorCode 
)

Appends the second string to the first string (merging them at the boundary) and returns the length of the first string.

The result is normalized if both the strings were normalized. The first and second strings must be different buffers.

Parameters:
norm2 UNormalizer2 instance
first string, should be normalized
firstLength length of the first string, or -1 if NUL-terminated
firstCapacity number of UChars that can be written to first
second string, should be normalized
secondLength length of the source string, or -1 if NUL-terminated
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
first
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
void unorm2_close ( UNormalizer2 norm2  ) 

Closes a UNormalizer2 instance from unorm2_openFiltered().

Do not close instances from unorm2_getInstance()!

Parameters:
norm2 UNormalizer2 instance to be closed
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
const UNormalizer2* unorm2_getInstance ( const char *  packageName,
const char *  name,
UNormalization2Mode  mode,
UErrorCode pErrorCode 
)

Returns a UNormalizer2 instance which uses the specified data file (packageName/name similar to ucnv_openPackage() and ures_open()/ResourceBundle) and which composes or decomposes text according to the specified mode.

Returns an unmodifiable singleton instance. Do not delete it.

Use packageName=NULL for data files that are part of ICU's own data. Use name="nfc" and UNORM2_COMPOSE/UNORM2_DECOMPOSE for Unicode standard NFC/NFD. Use name="nfkc" and UNORM2_COMPOSE/UNORM2_DECOMPOSE for Unicode standard NFKC/NFKD. Use name="nfkc_cf" and UNORM2_COMPOSE for Unicode standard NFKC_CF=NFKC_Casefold.

Parameters:
packageName NULL for ICU built-in data, otherwise application data package name
name "nfc" or "nfkc" or "nfkc_cf" or name of custom data file
mode normalization mode (compose or decompose etc.)
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
the requested UNormalizer2, if successful
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UBool unorm2_hasBoundaryAfter ( const UNormalizer2 norm2,
UChar32  c 
)

Tests if the character always has a normalization boundary after it, regardless of context.

For details see the Normalizer2 base class documentation.

Parameters:
norm2 UNormalizer2 instance
c character to test
Returns:
TRUE if c has a normalization boundary after it
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UBool unorm2_hasBoundaryBefore ( const UNormalizer2 norm2,
UChar32  c 
)

Tests if the character always has a normalization boundary before it, regardless of context.

For details see the Normalizer2 base class documentation.

Parameters:
norm2 UNormalizer2 instance
c character to test
Returns:
TRUE if c has a normalization boundary before it
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UBool unorm2_isInert ( const UNormalizer2 norm2,
UChar32  c 
)

Tests if the character is normalization-inert.

For details see the Normalizer2 base class documentation.

Parameters:
norm2 UNormalizer2 instance
c character to test
Returns:
TRUE if c is normalization-inert
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UBool unorm2_isNormalized ( const UNormalizer2 norm2,
const UChar s,
int32_t  length,
UErrorCode pErrorCode 
)

Tests if the string is normalized.

Internally, in cases where the quickCheck() method would return "maybe" (which is only possible for the two COMPOSE modes) this method resolves to "yes" or "no" to provide a definitive result, at the cost of doing more work in those cases.

Parameters:
norm2 UNormalizer2 instance
s input string
length length of the string, or -1 if NUL-terminated
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
TRUE if s is normalized
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
int32_t unorm2_normalize ( const UNormalizer2 norm2,
const UChar src,
int32_t  length,
UChar dest,
int32_t  capacity,
UErrorCode pErrorCode 
)

Writes the normalized form of the source string to the destination string (replacing its contents) and returns the length of the destination string.

The source and destination strings must be different buffers.

Parameters:
norm2 UNormalizer2 instance
src source string
length length of the source string, or -1 if NUL-terminated
dest destination string; its contents is replaced with normalized src
capacity number of UChars that can be written to dest
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
dest
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
int32_t unorm2_normalizeSecondAndAppend ( const UNormalizer2 norm2,
UChar first,
int32_t  firstLength,
int32_t  firstCapacity,
const UChar second,
int32_t  secondLength,
UErrorCode pErrorCode 
)

Appends the normalized form of the second string to the first string (merging them at the boundary) and returns the length of the first string.

The result is normalized if the first string was normalized. The first and second strings must be different buffers.

Parameters:
norm2 UNormalizer2 instance
first string, should be normalized
firstLength length of the first string, or -1 if NUL-terminated
firstCapacity number of UChars that can be written to first
second string, will be normalized
secondLength length of the source string, or -1 if NUL-terminated
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
first
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UNormalizer2* unorm2_openFiltered ( const UNormalizer2 norm2,
const USet filterSet,
UErrorCode pErrorCode 
)

Constructs a filtered normalizer wrapping any UNormalizer2 instance and a filter set.

Both are aliased and must not be modified or deleted while this object is used. The filter set should be frozen; otherwise the performance will suffer greatly.

Parameters:
norm2 wrapped Normalizer2 instance
filterSet USet which determines the characters to be normalized
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
the requested UNormalizer2, if successful
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
UNormalizationCheckResult unorm2_quickCheck ( const UNormalizer2 norm2,
const UChar s,
int32_t  length,
UErrorCode pErrorCode 
)

Tests if the string is normalized.

For the two COMPOSE modes, the result could be "maybe" in cases that would take a little more work to resolve definitively. Use spanQuickCheckYes() and normalizeSecondAndAppend() for a faster combination of quick check + normalization, to avoid re-checking the "yes" prefix.

Parameters:
norm2 UNormalizer2 instance
s input string
length length of the string, or -1 if NUL-terminated
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
UNormalizationCheckResult
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
int32_t unorm2_spanQuickCheckYes ( const UNormalizer2 norm2,
const UChar s,
int32_t  length,
UErrorCode pErrorCode 
)

Returns the end of the normalized substring of the input string.

In other words, with end=spanQuickCheckYes(s, ec); the substring UnicodeString(s, 0, end) will pass the quick check with a "yes" result.

The returned end index is usually one or more characters before the "no" or "maybe" character: The end index is at a normalization boundary. (See the class documentation for more about normalization boundaries.)

When the goal is a normalized string and most input strings are expected to be normalized already, then call this method, and if it returns a prefix shorter than the input string, copy that prefix and use normalizeSecondAndAppend() for the remainder.

Parameters:
norm2 UNormalizer2 instance
s input string
length length of the string, or -1 if NUL-terminated
pErrorCode Standard ICU error code. Its input value must pass the U_SUCCESS() test, or else the function returns immediately. Check for U_FAILURE() on output or use with function chaining. (See User Guide for details.)
Returns:
"yes" span end index
Draft:
This API may be changed in the future versions and was introduced in ICU 4.4
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

Generated on Sat Jan 23 15:17:39 2010 for ICU 4.3.4 by  doxygen 1.6.1