#include
The strtok() function returns a pointer to the next token in the string pointed to by the parameter. str1. The characters that make up the string addressed by the parameter str2, are delimiters that define a token. If there is no token to return, a null pointer is returned.
In the C99 version, the parameters str1 And str2 the restrict qualifier applied.
To split some string into tokens, the first time the strtok() function is called, the parameter str1 must point to the beginning of this line. On subsequent function calls as a parameter str1 a null pointer must be used. In this way, the entire string is tokenized.
Each call to the strtok() function can use a different set of delimiters.
This program breaks the string "The grass is green, the sun is shining" into tokens separated by spaces and commas. The result will be
Grass|Green|Sunshine|Shines #include
Programming languages may include special functions to work with strings, thereby saving the programmer from having to write their own string processing functions. For example, it is often necessary to determine the length of a string, and so languages provide a function to measure its length.
In the C programming language, functions for working with strings are declared in the string.h header file, which you must remember to include in your source code. There are about twenty functions for working with strings. Among them are those that search for characters in a string, comparison functions, string copying, as well as more specific ones. List and description of most of the existing this moment in the C language, functions can be found in the appendix of the book by B. Kernighan, D. Ritchie "The C Programming Language. Second Edition".
All functions declared in string.h may or may not modify one of the strings passed by the pointer in the course of their work. It depends on the purpose of the function. However, most of them return something: either a pointer to a character or an integer. Moreover, if the function changes one of its parameters and was called for this, then what it returns can be ignored (that is, not assigned to anything in the calling function).
For example, the strcpy() function has the following declaration: char *strcpy (char *, const char*) . It copies the string pointed to by the second parameter to the string pointed to by the first parameter. So the first parameter is changed. In addition, the function returns a pointer to the first character of the string:
char s1[ 10 ] , s2[ 10 ] ; char*s3; s3 = s2; gets (s1) ; s3 = strcpy (s2, s1) ; puts (s2) ; puts (s3) ; printf("%p, %p \n", s2, s3) ;
Here s2 and s3 point to the same character (printf() outputs the same address). However, what strcpy() returns cannot be assigned to an array. The result of this function is usually not assigned to anything; sometimes it is enough that it simply changes one of the strings passed by the pointer.
Another thing is functions such as strlen() or strcmp() , which do not change parameters, but are called for the sake of the result. The strcmp() function compares the two argument strings letter by letter (lexicographically) and returns 0, -1, or 1. For example, calling strcmp("boy", "body") will return 1 because letter code "y" more than a letter"d". Calling strcmp("body", "boy") will return -1, because the first argument is lexicographically less than the second.
Using the strtok() function, you can split a string into separate parts (tokens). The declaration of this function looks like this char *strtok (char *, const char *) . The first time the function is called, the first parameter is the string to split. The second parameter specifies the delimiter string. On subsequent function calls for the same row, the first parameter must be NULL, because the function has already "remembered" what it works with. Consider an example:
char str = "one, two, three, four, five" ; char*sp; sp = strtok (str, ", " ) ; while (sp) ( puts (sp) ; sp = strtok (NULL, ", " ) ; )
As a result of executing this code, the following words are displayed in a column:
one two three four five
The first time strtok() is called, the function is passed a pointer to the first character of the array and a delimiter string. After this call, the array str is changed, only the word "one" remains in it, and the function also returns a pointer to this word, which is assigned to sp.
Although we have lost the remainder of the array in the calling function, a pointer to the remainder of the array is retained inside strtok(). When NULL is passed, the function "knows" to work with that tail.
When you just need to join two strings, the problem is easily solved by calling the strcat() function, which appends the second to the end of the first argument. A similar function, strncat(), appends n characters from the second string to the first. n is specified as the third parameter.
What if the situation is more complex? For example, there are two non-empty lines and you need to connect the beginning of the first and the end of the second. You can do this using the strcpy() function, if you pass references not to the first characters of strings:
char s1[ 20 ] = "Peter Smith" , s2 = "Julia Roberts" ; strcpy (s1+ 5 , s2+ 5 ) ; puts(s1) ;
IN this case"Peter Roberts" will be displayed on the screen. Why did it happen? A pointer to the sixth character of the first string was passed to the strcpy() function. This led to the fact that when copying, the characters of this string are overwritten only starting from the 6th, because strcpy() doesn't "know" anything about previous characters. Only part of the string is also passed as the second argument, which is copied into the first.
How to insert one line in the middle of another? You can solve this problem using the third "buffer" line, where you can first copy the first line, then the second, overwriting the end of the first, then attach the end of the first. But you can also do this:
char s1[ 20 ] = "one three" , s2[ 20 ] = "two" ; strcpy (s2+ 3 , s1+ 3 ) ; strcpy (s1+ 4 , s2) ; puts(s1) ;
Here, first, the end of the first is copied into the second line, it turns out "two three". Then the second line is copied to the first line, bypassing its beginning.
Exercise
Below are descriptions of some functions that perform operations on strings. Design and write small programs that illustrate how these functions work.
Syntax:
#include
char *strtok(char *str, const char *sep);
Arguments:
str is a pointer to the string to be split.
sep is a pointer to a string containing a set of delimiter characters.
Return value:
NULL - if the string str cannot be divided into parts.
Pointer to the first character of the selected part of the string.
Description:
The strtok function extracts the next part of the string pointed to by the str argument, separated by one of the delimiter characters specified in the string pointed to by the sep argument. A successive call to the strtok function results in splitting the string str into parts (tokens).
“The first call to the strtok function specifies the beginning of a string to be separated (str) and the beginning of a string containing delimiters (sep). At first strok function iterates through the characters of the string str one by one and looks for a character not contained in the delimiter string sep. If in the string str the end-of-line character is encountered before a character not included in the string sep was found, then the string str cannot be divided into parts and a null pointer (NULL) is returned. If such a character is found, it is considered to be the beginning of the first part of str."
Next, the strtok function looks for the delimiter, that is, the character included in the string sep. If no such character is found, then the string str is considered to be one part, and subsequent splits of the string str will return a null pointer. If such a character is found. then it is replaced with a null character (the end-of-line character). Next, the strtok function remembers the current position (a pointer to the character from which the search for the next part of the string will begin) and returns a pointer to the beginning of the first selected part of the string.
If the strtok function returned a non-null pointer, you can continue splitting str into pieces. To continue splitting the string, the strtok function is called again, but instead of a pointer to the string to be split, NULL is specified as the first augment. In this case, the strtok function will continue splitting from the remembered address. The partitioning algorithm will remain the same.
Example:
In the example, the string "test1/test2/test3/test4" is split into parts on the delimiter "/" using the strtok function. The split result is printed to the console.
Result:
Console output:
4 answers
Two things to know about strtok . As mentioned, it "maintains internal state". In addition, he messed up the line you feed her. Essentially it will write "\0" where it will find the marker you provided and return a pointer to the beginning of the string. Internally, it maintains the location of the last token; and the next time you call it, it will start from there.
An important consequence is that you cannot use strtok on a string like const char* "hello world"; , since you'll get an access violation when changing the contents of a const char* string.
The "good thing" about strtok is that it doesn't actually copy strings, so you don't have to manage extra memory allocation, etc. But if you don't understand the above, you will have trouble using it.
Example. If you have "this, is, string", successive calls to strtok will generate pointers as follows (the ^ value is the return value). Note that "\0" is added where the tokens are found; this means that the original line is changed:
T h i s , i s , a , s t r i n g \0 this,is,a,string t h i s \0 i s , a , s t r i n g \0 this ^ t h i s \0 i s \0 a , s t r i n g \0 is ^ t h i s \0 i s \0 a \ 0 s t r i n g \0 a ^ t h i s \0 i s \0 a \0 s t r i n g \0 string ^
Hope this makes sense.
The strtok() function stores data between calls. It uses this data when you call it with a NULL pointer.
The point at which the last token was found is stored internally with a function to be used on the next call (no specific library implementation required to prevent data crashes).
strtok maintains internal state. When you call it with non-NULL, it reinitializes itself to use the string you supply. When you call it with NULL , it uses that string and any other state it currently has to return the next token.
Due to the way strtok works, you need to make sure you are linking to the multi-threaded version of the C runtime if you are writing a multi-threaded application. This ensures that each thread gets its own internal state for strtok .
The strtok function stores data in an internal static variable that is shared among all threads.
For thread safety you should use strtok_r
Take a look at static char *last;
Char * strtok(s, delim) register char *s; register const char *delim; ( register char *spanp; register int c, sc; char *tok; static char *last; if (s == NULL && (s = last) == NULL) return (NULL); /* * Skip (span) leading delimiters (s += strspn(s, delim), sort of).*/ cont: c = *s++; for (spanp = (char *)delim; (sc = *spanp++) != 0;) ( if (c == sc) goto cont; ) if (c == 0) ( /* no non-delimiter characters */ last = NULL; return (NULL); ) tok = s - 1; /* * Scan token (scan for delimiters : s += strcspn(s, delim), sort of). * Note that delim must have one NUL; we stop if we see that, too. */ for (;;) ( c = *s++; spanp = (char *)delim; do ( if ((sc = *spanp++) == c) ( if (c == 0) s = NULL; else s[-1] = 0; last = s; return (tok); ) ) while (sc != 0); ) /* NOTREACHED */ )
share1) Finds the next token in the null-terminated byte string pointed to by str . Delimiter characters are identified by the null byte string pointed to by delim .
This function is called multiple multiplication to get consecutive tokens from the same string.
- If str! = NULL str ! = NULL , the call is treated as the first call to strtok for that particular row. The function searches for the first character that Not contained in delim .
- If such a symbol was not found, then there are no tokens in it, and the function returns a null pointer.
- If such a character was found, it will be token start. The function then searches from that point for the first character contained in delim .
- If no such character is found, str has only one token, and future calls to strtok return a null pointer
- If such a character was found, it is replaced the null character "\0" and the pointer to the next character is stored in a static location for subsequent calls.
- The function then returns a pointer to the beginning of the token
- If str == NULL , the call is treated like subsequent calls to strtok: the function continues from where it left off in the previous call. The behavior is the same as if the previously stored pointer were passed as str .
Behavior is undefined unless str or delim is a pointer to a null-terminated byte string.
2) Same as (1) except that each step writes the number of characters left to look up in str to *strmax and writes the internal state of the tokenizer to *ptr . Repeated calls (with null strmax) must pass strmax and ptr with the values stored by the previous call. In addition, at runtime, the following errors and the current one is called set function constraint handler without storing anything in the object pointed to by ptr
- strmax , delim or ptr - null pointer
- on a non-initial call (with a null str), *ptr is a null pointer
- on first call, *strmax is zero or greater than RSIZE_MAX
- end of token search reaches the end of the original string (as measured by the initial *strmax value) without encountering a null terminator
Behavior is undefined if both str points to a character array that does not have a null character, and strmax points to a value that is greater than the size of that character array. Like all bounds-checking-related functions, strtok_s is only guaranteed to be available if __STDC_LIB_EXT1__ is implementation-defined, and if the user defines __STDC_WANT_LIB_EXT1__ for the integer constant 1 before including string.h .
options
Return value
Returns a pointer to the start of the next token, or NULL if there are no more tokens.
The note
This function is destructive: it writes the characters "\0" in the elements of the string str . In particular, a string literal cannot be used as the first argument to strtok .
Each call to strtok modifies a static variable: not thread safe.
Unlike most other tokenizers, the delimiters in strtok can be different for each successive token and can even depend on the contents of previous tokens.
The strtok_s function differs from the POSIX strtok_r function by storing it outside of the tokenized string and checking run-time limits.
example
#define __STDC_WANT_LIB_EXT1__ 1 #include
#include int main(void) ( char input = "A bird came down the walk"; printf("Parsing the input string "%s"\n", input); char *token = strtok(input, " "); while( token) ( puts(token); token = strtok(NULL, " "); ) printf("Contents of the input string now: ""); for(size_t n = 0; n< sizeof input; ++n) input[n] ? printf("%c", input[n]) : printf("\\0"); puts("""); #ifdef __STDC_LIB_EXT1__ char str = "A bird came down the walk"; rsize_t strmax = sizeof str; const char *delim = " "; char *next_token; printf("Parsing the input string "%s"\n", str); token = strtok_s(str, &strmax, delim, &next_token); while(token) { puts(token); token = strtok_s(NULL, &strmax, delim, &next_token); } printf("Contents of the input string now: ""); for(size_t n = 0; n < sizeof str; ++n) str[n] ? printf("%c", str[n]) : printf("\\0"); puts("""); #endif } Possible output:
Parsing the input string "A bird came down the walk" A bird came down the walk Contents of the input string now: "A\0bird\0came\0down\0the\0walk\0" Parsing the input string "A bird came down the walk" A bird came down the walk Contents of the input string now: "A\0bird\0came\0down\0the\0walk\0"
- C11 (ISO/IEC 9899:2011):
- 7.24.5.8 The strtok function (p: 369-370)
- K.3.7.3.1 strtok_s function (p: 620-621)
- C99 (ISO/IEC 9899:1999):
- 7.21.5.8 The strtok function (p: 332-333)
- C89/C90 (ISO/IEC 9899:1990):
- 4.11.5.8 strtok function
finds the first location of any character in one string, in another string
(function)
only characters not found in another byte string
(function)returns the length of the maximum initial segment, which consists of
only characters found in another byte string
(function)(C95) (C11)
finds the next token in a wide string
(function)C++ Documentation for strtok Thematic materials:Updated: 05/24/2021103583If you notice an error, select a piece of text and press Ctrl + Enter
