uspoof.h File Reference

C API: Unicode Spoof Detection. More...

#include "unicode/utypes.h"
#include "unicode/uset.h"
#include "unicode/parseerr.h"
#include "unicode/localpointer.h"

Go to the source code of this file.

Typedefs

typedef struct USpoofChecker USpoofChecker
 typedef for C of USpoofChecker

Enumerations

enum  USpoofChecks {
  USPOOF_SINGLE_SCRIPT_CONFUSABLE = 1, USPOOF_MIXED_SCRIPT_CONFUSABLE = 2, USPOOF_WHOLE_SCRIPT_CONFUSABLE = 4, USPOOF_ANY_CASE = 8,
  USPOOF_SINGLE_SCRIPT = 16, USPOOF_INVISIBLE = 32, USPOOF_CHAR_LIMIT = 64, USPOOF_ALL_CHECKS = 0x7f
}
 

Enum for the kinds of checks that USpoofChecker can perform.

More...

Functions

USpoofCheckeruspoof_open (UErrorCode *status)
 Create a Unicode Spoof Checker, configured to perform all checks except for USPOOF_LOCALE_LIMIT and USPOOF_CHAR_LIMIT.
U_CAPI USpoofCheckeruspoof_openFromSerialized (const void *data, int32_t length, int32_t *pActualLength, UErrorCode *pErrorCode)
 Open a Spoof checker from its serialized from, stored in 32-bit-aligned memory.
U_CAPI USpoofCheckeruspoof_openFromSource (const char *confusables, int32_t confusablesLen, const char *confusablesWholeScript, int32_t confusablesWholeScriptLen, int32_t *errType, UParseError *pe, UErrorCode *status)
 Open a Spoof Checker from the source form of the spoof data.
void uspoof_close (USpoofChecker *sc)
 Close a Spoof Checker, freeing any memory that was being held by its implementation.
USpoofCheckeruspoof_clone (const USpoofChecker *sc, UErrorCode *status)
 Clone a Spoof Checker.
void uspoof_setChecks (USpoofChecker *sc, int32_t checks, UErrorCode *status)
 Specify the set of checks that will be performed by the check functions of this Spoof Checker.
int32_t uspoof_getChecks (const USpoofChecker *sc, UErrorCode *status)
 Get the set of checks that this Spoof Checker has been configured to perform.
void uspoof_setAllowedLocales (USpoofChecker *sc, const char *localesList, UErrorCode *status)
 Limit characters that are acceptable in identifiers being checked to those normally used with the languages associated with the specified locales.
const char * uspoof_getAllowedLocales (USpoofChecker *sc, UErrorCode *status)
 Get a list of locales for the scripts that are acceptable in strings to be checked.
void uspoof_setAllowedChars (USpoofChecker *sc, const USet *chars, UErrorCode *status)
 Limit the acceptable characters to those specified by a Unicode Set.
const USetuspoof_getAllowedChars (const USpoofChecker *sc, UErrorCode *status)
 Get a USet for the characters permitted in an identifier.
int32_t uspoof_check (const USpoofChecker *sc, const UChar *text, int32_t length, int32_t *position, UErrorCode *status)
 Check the specified string for possible security issues.
int32_t uspoof_checkUTF8 (const USpoofChecker *sc, const char *text, int32_t length, int32_t *position, UErrorCode *status)
 Check the specified string for possible security issues.
int32_t uspoof_areConfusable (const USpoofChecker *sc, const UChar *s1, int32_t length1, const UChar *s2, int32_t length2, UErrorCode *status)
 Check the whether two specified strings are visually confusable.
int32_t uspoof_areConfusableUTF8 (const USpoofChecker *sc, const char *s1, int32_t length1, const char *s2, int32_t length2, UErrorCode *status)
 Check the whether two specified strings are visually confusable.
int32_t uspoof_getSkeleton (const USpoofChecker *sc, uint32_t type, const UChar *s, int32_t length, UChar *dest, int32_t destCapacity, UErrorCode *status)
 Get the "skeleton" for an identifier string.
int32_t uspoof_getSkeletonUTF8 (const USpoofChecker *sc, uint32_t type, const char *s, int32_t length, char *dest, int32_t destCapacity, UErrorCode *status)
 Get the "skeleton" for an identifier string.
U_CAPI int32_t uspoof_serialize (USpoofChecker *sc, void *data, int32_t capacity, UErrorCode *status)
 Serialize the data for a spoof detector into a chunk of memory.

Detailed Description

C API: Unicode Spoof Detection.

Definition in file uspoof.h.


Enumeration Type Documentation

Enum for the kinds of checks that USpoofChecker can perform.

These enum values are used both to select the set of checks that will be performed, and to report results from the check function.

Stable:
ICU 4.4
Enumerator:
USPOOF_SINGLE_SCRIPT_CONFUSABLE 

Single script confusable test.

When testing whether two identifiers are confusable, report that they are if both are from the same script and they are visually confusable. Note: this test is not applicable to a check of a single identifier.

USPOOF_MIXED_SCRIPT_CONFUSABLE 

Mixed script confusable test.

When checking a single identifier, report a problem if the identifier contains multiple scripts, and is confusable with some other identifer in a single script When testing whether two identifiers are confusable, report that they are if the two IDs are visually confusable, and and at least one contains characters from more than one script.

USPOOF_WHOLE_SCRIPT_CONFUSABLE 

Whole script confusable test.

When checking a single identifier, report a problem if The identifier is of a single script, and there exists a confusable identifier in another script. When testing whether two identfiers are confusable, report that they are if each is of a single script, the scripts of the two identifiers are different, and the identifiers are visually confusable.

USPOOF_ANY_CASE 

Any Case Modifier for confusable identifier tests.

If specified, consider all characters, of any case, when looking for confusables. If USPOOF_ANY_CASE is not specified, identifiers being checked are assumed to have been case folded. Upper case conusable characters will not be checked. Selects between Lower Case Confusable and Any Case Confusable.

USPOOF_SINGLE_SCRIPT 

Check that an identifer contains only characters from a single script (plus chars from the common and inherited scripts.

) Applies to checks of a single identifier check only.

USPOOF_INVISIBLE 

Check an identifier for the presence of invisble characters, such as zero-width spaces, or character sequences that are likely not to display, such as multiple occurences of the same non-spacing mark.

This check does not test the input string as a whole for conformance to any particular syntax for identifiers.

USPOOF_CHAR_LIMIT 

Check that an identifier contains only characters from a specified set of acceptable characters.

See uspoof_setAllowedChars() and uspoof_setAllowedLocales().

Definition at line 148 of file uspoof.h.


Function Documentation

int32_t uspoof_areConfusable ( const USpoofChecker sc,
const UChar s1,
int32_t  length1,
const UChar s2,
int32_t  length2,
UErrorCode status 
)

Check the whether two specified strings are visually confusable.

The types of confusability to be tested - single script, mixed script, or whole script - are determined by the check options set for the USpoofChecker.

The tests to be performed are controlled by the flags USPOOF_SINGLE_SCRIPT_CONFUSABLE USPOOF_MIXED_SCRIPT_CONFUSABLE USPOOF_WHOLE_SCRIPT_CONFUSABLE At least one of these tests must be selected.

USPOOF_ANY_CASE is a modifier for the tests. Select it if the identifiers may be of mixed case. If identifiers are case folded for comparison and display to the user, do not select the USPOOF_ANY_CASE option.

Parameters:
sc The USpoofChecker
s1 The first of the two strings to be compared for confusability. The strings are in UTF-16 format.
length1 the length of the first string, expressed in 16 bit UTF-16 code units, or -1 if the string is zero terminated.
s2 The second of the two strings to be compared for confusability. The strings are in UTF-16 format.
length2 The length of the second string, expressed in 16 bit UTF-16 code units, or -1 if the string is zero terminated.
status The error code, set if an error occured while attempting to perform the check. Confusability of the strings is not reported here, but through this function's return value.
Returns:
An integer value with bit(s) set corresponding to the type of confusability found, as defined by enum USpoofChecks. Zero is returned if the strings are not confusable.
Stable:
ICU 4.4
int32_t uspoof_areConfusableUTF8 ( const USpoofChecker sc,
const char *  s1,
int32_t  length1,
const char *  s2,
int32_t  length2,
UErrorCode status 
)

Check the whether two specified strings are visually confusable.

The types of confusability to be tested - single script, mixed script, or whole script - are determined by the check options set for the USpoofChecker.

Parameters:
sc The USpoofChecker
s1 The first of the two strings to be compared for confusability. The strings are in UTF-8 format.
length1 the length of the first string, in bytes, or -1 if the string is zero terminated.
s2 The second of the two strings to be compared for confusability. The strings are in UTF-18 format.
length2 The length of the second string in bytes, or -1 if the string is zero terminated.
status The error code, set if an error occured while attempting to perform the check. Confusability of the strings is not reported here, but through this function's return value.
Returns:
An integer value with bit(s) set corresponding to the type of confusability found, as defined by enum USpoofChecks. Zero is returned if the strings are not confusable.
Stable:
ICU 4.4
int32_t uspoof_check ( const USpoofChecker sc,
const UChar text,
int32_t  length,
int32_t *  position,
UErrorCode status 
)

Check the specified string for possible security issues.

The text to be checked will typically be an indentifier of some sort. The set of checks to be performed is specified with uspoof_setChecks().

Parameters:
sc The USpoofChecker
text The string to be checked for possible security issues, in UTF-16 format.
length the length of the string to be checked, expressed in 16 bit UTF-16 code units, or -1 if the string is zero terminated.
position An out parameter that receives the index of the first string position that fails the allowed character limitation checks. This parameter may be null if the position information is not needed. If the string passes the requested checks the parameter value will not be set.
status The error code, set if an error occured while attempting to perform the check. Spoofing or security issues detected with the input string are not reported here, but through the function's return value.
Returns:
An integer value with bits set for any potential security or spoofing issues detected. The bits are defined by enum USpoofChecks. Zero is returned if no issues are found with the input string.
Stable:
ICU 4.4
int32_t uspoof_checkUTF8 ( const USpoofChecker sc,
const char *  text,
int32_t  length,
int32_t *  position,
UErrorCode status 
)

Check the specified string for possible security issues.

The text to be checked will typically be an indentifier of some sort. The set of checks to be performed is specified with uspoof_setChecks().

Parameters:
sc The USpoofChecker
text A UTF-8 string to be checked for possible security issues.
length the length of the string to be checked, or -1 if the string is zero terminated.
position An out parameter that receives the index of the first string position that fails the allowed character limitation checks. This parameter may be null if the position information is not needed. If the string passes the requested checks the parameter value will not be set.
status The error code, set if an error occured while attempting to perform the check. Spoofing or security issues detected with the input string are not reported here, but through the function's return value. If the input contains invalid UTF-8 sequences, a status of U_INVALID_CHAR_FOUND will be returned.
Returns:
An integer value with bits set for any potential security or spoofing issues detected. The bits are defined by enum USpoofChecks. Zero is returned if no issues are found with the input string.
Stable:
ICU 4.4
USpoofChecker* uspoof_clone ( const USpoofChecker sc,
UErrorCode status 
)

Clone a Spoof Checker.

The clone will be set to perform the same checks as the original source.

Parameters:
sc The source USpoofChecker
status The error code, set if this function encounters a problem.
Returns:
Stable:
ICU 4.4
void uspoof_close ( USpoofChecker sc  ) 

Close a Spoof Checker, freeing any memory that was being held by its implementation.

Stable:
ICU 4.4
const USet* uspoof_getAllowedChars ( const USpoofChecker sc,
UErrorCode status 
)

Get a USet for the characters permitted in an identifier.

This corresponds to the limits imposed by the Set Allowed Characters functions. Limitations imposed by other checks will not be reflected in the set returned by this function.

The returned set will be frozen, meaning that it cannot be modified by the caller.

Ownership of the returned set remains with the Spoof Detector. The returned set will become invalid if the spoof detector is closed, or if a new set of allowed characters is specified.

Parameters:
sc The USpoofChecker
status The error code, set if this function encounters a problem.
Returns:
A USet containing the characters that are permitted by the USPOOF_CHAR_LIMIT test.
Stable:
ICU 4.4
const char* uspoof_getAllowedLocales ( USpoofChecker sc,
UErrorCode status 
)

Get a list of locales for the scripts that are acceptable in strings to be checked.

If no limitations on scripts have been specified, an empty string will be returned.

uspoof_setAllowedChars() will reset the list of allowed to be empty.

The format of the returned list is the same as that supplied to uspoof_setAllowedLocales(), but returned list may not be identical to the originally specified string; the string may be reformatted, and information other than languages from the originally specified locales may be omitted.

Parameters:
sc The USpoofChecker
status The error code, set if this function encounters a problem.
Returns:
A string containing a list of locales corresponding to the acceptable scripts, formatted like an HTTP Accept Language value.
Stable:
ICU 4.4
int32_t uspoof_getChecks ( const USpoofChecker sc,
UErrorCode status 
)

Get the set of checks that this Spoof Checker has been configured to perform.

Parameters:
sc The USpoofChecker
status The error code, set if this function encounters a problem.
Returns:
The set of checks that this spoof checker will perform. The value is a bit set, obtained by OR-ing together values from enum USpoofChecks.
Stable:
ICU 4.4
int32_t uspoof_getSkeleton ( const USpoofChecker sc,
uint32_t  type,
const UChar s,
int32_t  length,
UChar dest,
int32_t  destCapacity,
UErrorCode status 
)

Get the "skeleton" for an identifier string.

Skeletons are a transformation of the input string; Two strings are confusable if their skeletons are identical. See Unicode UAX 39 for additional information.

Using skeletons directly makes it possible to quickly check whether an identifier is confusable with any of some large set of existing identifiers, by creating an efficiently searchable collection of the skeletons.

Parameters:
sc The USpoofChecker
type The type of skeleton, corresponding to which of the Unicode confusable data tables to use. The default is Mixed-Script, Lowercase. Allowed options are USPOOF_SINGLE_SCRIPT_CONFUSABLE and USPOOF_ANY_CASE_CONFUSABLE. The two flags may be ORed.
s The input string whose skeleton will be computed.
length The length of the input string, expressed in 16 bit UTF-16 code units, or -1 if the string is zero terminated.
dest The output buffer, to receive the skeleton string.
destCapacity The length of the output buffer, in 16 bit units. The destCapacity may be zero, in which case the function will return the actual length of the skeleton.
status The error code, set if an error occured while attempting to perform the check.
Returns:
The length of the skeleton string. The returned length is always that of the complete skeleton, even when the supplied buffer is too small (or of zero length)
Stable:
ICU 4.4
int32_t uspoof_getSkeletonUTF8 ( const USpoofChecker sc,
uint32_t  type,
const char *  s,
int32_t  length,
char *  dest,
int32_t  destCapacity,
UErrorCode status 
)

Get the "skeleton" for an identifier string.

Skeletons are a transformation of the input string; Two strings are confusable if their skeletons are identical. See Unicode UAX 39 for additional information.

Using skeletons directly makes it possible to quickly check whether an identifier is confusable with any of some large set of existing identifiers, by creating an efficiently searchable collection of the skeletons.

Parameters:
sc The USpoofChecker
type The type of skeleton, corresponding to which of the Unicode confusable data tables to use. The default is Mixed-Script, Lowercase. Allowed options are USPOOF_SINGLE_SCRIPT_CONFUSABLE and USPOOF_ANY_CASE. The two flags may be ORed.
s The UTF-8 format input string whose skeleton will be computed.
length The length of the input string, in bytes, or -1 if the string is zero terminated.
dest The output buffer, to receive the skeleton string.
destCapacity The length of the output buffer, in bytes. The destCapacity may be zero, in which case the function will return the actual length of the skeleton.
status The error code, set if an error occured while attempting to perform the check. Possible Errors include U_INVALID_CHAR_FOUND for invalid UTF-8 sequences, and U_BUFFER_OVERFLOW_ERROR if the destination buffer is too small to hold the complete skeleton.
Returns:
The length of the skeleton string, in bytes. The returned length is always that of the complete skeleton, even when the supplied buffer is too small (or of zero length)
Stable:
ICU 4.4
USpoofChecker* uspoof_open ( UErrorCode status  ) 

Create a Unicode Spoof Checker, configured to perform all checks except for USPOOF_LOCALE_LIMIT and USPOOF_CHAR_LIMIT.

Note that additional checks may be added in the future, resulting in the changes to the default checking behavior.

Parameters:
status The error code, set if this function encounters a problem.
Returns:
the newly created Spoof Checker
Stable:
ICU 4.4
U_CAPI USpoofChecker* uspoof_openFromSerialized ( const void *  data,
int32_t  length,
int32_t *  pActualLength,
UErrorCode pErrorCode 
)

Open a Spoof checker from its serialized from, stored in 32-bit-aligned memory.

Inverse of uspoof_serialize(). The memory containing the serailized data must remain valid and unchanged as long as the spoof checker, or any cloned copies of the spoof checker, are in use. Ownership of the memory remains with the caller. The spoof checker (and any clones) must be closed prior to deleting the serialized data.

Parameters:
data a pointer to 32-bit-aligned memory containing the serialized form of spoof data
length the number of bytes available at data; can be more than necessary
pActualLength receives the actual number of bytes at data taken up by the data; can be NULL
pErrorCode ICU error code
Returns:
the spoof checker.
See also:
uspoof_open
uspoof_serialize
Stable:
ICU 4.4
U_CAPI USpoofChecker* uspoof_openFromSource ( const char *  confusables,
int32_t  confusablesLen,
const char *  confusablesWholeScript,
int32_t  confusablesWholeScriptLen,
int32_t *  errType,
UParseError pe,
UErrorCode status 
)

Open a Spoof Checker from the source form of the spoof data.

The Three inputs correspond to the Unicode data files confusables.txt confusablesWholeScript.txt and xidmdifications.txt as described in Unicode UAX 39. The syntax of the source data is as described in UAX 39 for these files, and the content of these files is acceptable input.

The character encoding of the (char *) input text is UTF-8.

Parameters:
confusables a pointer to the confusable characters definitions, as found in file confusables.txt from unicode.org.
confusablesLen The length of the confusables text, or -1 if the input string is zero terminated.
confusablesWholeScript a pointer to the whole script confusables definitions, as found in the file xonfusablesWholeScript.txt from unicode.org.
confusablesWholeScriptLen The length of the whole script confusables text, or -1 if the input string is zero terminated.
errType In the event of an error in the input, indicates which of the input files contains the error. The value is one of USPOOF_SINGLE_SCRIPT_CONFUSABLE or USPOOF_WHOLE_SCRIPT_CONFUSABLE, or zero if no errors are found.
pe In the event of an error in the input, receives the position in the input text (line, offset) of the error.
status an in/out ICU UErrorCode. Among the possible errors is U_PARSE_ERROR, which is used to report syntax errors in the input.
Returns:
A spoof checker that uses the rules from the input files.
Stable:
ICU 4.4
U_CAPI int32_t uspoof_serialize ( USpoofChecker sc,
void *  data,
int32_t  capacity,
UErrorCode status 
)

Serialize the data for a spoof detector into a chunk of memory.

The flattened spoof detection tables can later be used to efficiently instantiate a new Spoof Detector.

Parameters:
sc the Spoof Detector whose data is to be serialized.
data a pointer to 32-bit-aligned memory to be filled with the data, can be NULL if capacity==0
capacity the number of bytes available at data, or 0 for preflighting
status an in/out ICU UErrorCode; possible errors include:

  • U_BUFFER_OVERFLOW_ERROR if the data storage block is too small for serialization
  • U_ILLEGAL_ARGUMENT_ERROR the data or capacity parameters are bad
Returns:
the number of bytes written or needed for the spoof data
See also:
utrie2_openFromSerialized()
Stable:
ICU 4.4
void uspoof_setAllowedChars ( USpoofChecker sc,
const USet chars,
UErrorCode status 
)

Limit the acceptable characters to those specified by a Unicode Set.

Any previously specified character limit is is replaced by the new settings. This includes limits on characters that were set with the uspoof_setAllowedLocales() function.

The USPOOF_CHAR_LIMIT test is automatically enabled for this USpoofChecker by this function.

Parameters:
sc The USpoofChecker
chars A Unicode Set containing the list of charcters that are permitted. Ownership of the set remains with the caller. The incoming set is cloned by this function, so there are no restrictions on modifying or deleting the USet after calling this function.
status The error code, set if this function encounters a problem.
Stable:
ICU 4.4
void uspoof_setAllowedLocales ( USpoofChecker sc,
const char *  localesList,
UErrorCode status 
)

Limit characters that are acceptable in identifiers being checked to those normally used with the languages associated with the specified locales.

Any previously specified list of locales is replaced by the new settings.

A set of languages is determined from the locale(s), and from those a set of acceptable Unicode scripts is determined. Characters from this set of scripts, along with characters from the "common" and "inherited" Unicode Script categories will be permitted.

Supplying an empty string removes all restrictions; characters from any script will be allowed.

The USPOOF_CHAR_LIMIT test is automatically enabled for this USpoofChecker when calling this function with a non-empty list of locales.

The Unicode Set of characters that will be allowed is accessible via the uspoof_getAllowedChars() function. uspoof_setAllowedLocales() will replace any previously applied set of allowed characters.

Adjustments, such as additions or deletions of certain classes of characters, can be made to the result of uspoof_setAllowedLocales() by fetching the resulting set with uspoof_getAllowedChars(), manipulating it with the Unicode Set API, then resetting the spoof detectors limits with uspoof_setAllowedChars()

Parameters:
sc The USpoofChecker
localesList A list list of locales, from which the language and associated script are extracted. The locales are comma-separated if there is more than one. White space may not appear within an individual locale, but is ignored otherwise. The locales are syntactically like those from the HTTP Accept-Language header. If the localesList is empty, no restrictions will be placed on the allowed characters.
status The error code, set if this function encounters a problem.
Stable:
ICU 4.4
void uspoof_setChecks ( USpoofChecker sc,
int32_t  checks,
UErrorCode status 
)

Specify the set of checks that will be performed by the check functions of this Spoof Checker.

Parameters:
sc The USpoofChecker
checks The set of checks that this spoof checker will perform. The value is a bit set, obtained by OR-ing together values from enum USpoofChecks.
status The error code, set if this function encounters a problem.
Stable:
ICU 4.4
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Friends Defines

Generated on Sat Jan 23 15:17:40 2010 for ICU 4.3.4 by  doxygen 1.6.1