Text processing

Table of contents

Description

This block serves various manipulations with text, which are very often required in practice. Process the parsed text, clean it of garbage, translate it into other languages - all this, and much more, can be done by a text processing "cube".

How to add an action to a project?

Via context menu Add Action -> Data -> Text Processing

Open

Or use smart search.

Where is word processing applied?

• Escape strings . To escape special characters

• Regex. Regular expression search for text

• Spintax. Randomize, uniqueize text

• Split. Split a string into several others using a separator

• ToChar. Convert Unicode code to character

• ToLower, ToUpper. Convert uppercase letters to lowercase and vice versa

• Trim. Clear text from extra whitespace characters

• UrlEncode, UrlDecode. Encode \ Decode URL

• To a variable, list, table . Put data in variable, list or table

• Replacement . Replace text

• Translation . Perform translation into another language

• Preparing JavaScript . Process text for use in a Logic (IF-ELSE) or JavaScript action

• Get substring

• Transliteration . Make text transliterated

How to use the action?

The properties window consists mainly of three areas:

1. The input string is text, a variable, or a combination of both.

2. Actions on the string, properties and their settings.

3. The output string (result) in a variable.

Open

Open

All possible operations with this "cube":

Escape sequences

Escaping sequences. This action replaces the characters * +? | {[() ^ $. # (places a "slash" in front of each specified character -\ ) This technique is often used to work with queries and for the regex engine to use these characters literally, rather than as commands or metacharacters.

Before Application: {"animal": "cat"}
After: \ {"animal": \ "cat"}

Regex

Open

Processing text with regular expressions.
Regulars are very convenient to parse strings to find the required substring for a given pattern. This action allows you to parse not only the first found value, but the entire group and save the values to variables or a table.

Input field “Regex”

In this field, you must enter a regular expression, which will be used to search the text. Example -(?<=<title>).*(?=</title>)

Error with an empty answer

If this setting is checked and the regular expression does not find anything in the text, then the action will fail (exit via the red branch).

What to get

First

The first match found will be saved to the variable.

All

Save all search results to a list.

One match

Keep only one match.
In the field that appears, you can enter the sequence number of the match (numbering from zero!) Or select the Last or Random (random) value.

Open

Match No.

Save to the list only the specified match numbers (numbering from zero !, specify separated by commas).

To variables

This function is used when working with group regular expressions. An example under the spoiler:

Click here to expand the example

Let's imagine that there is the following text:

21.01.2003, 11:34:00.9299 11.12.2013, 01:22:55.3021 04.01.2007, 08:00:06.0032

And the task is to disassemble it into its components. To do this, we will use the following regular expression: (\d{2}).(\d{2}).(\d{4}), (\d{2}):(\d{2}):(\d{2}).(\d{4})

This is how the output looks in the Regular Expression Tester:

Open

Let's imagine that we need to take in variables the day, month and year from the second row. Here's how you can do it:

Open

The match number in our case is the line number. Because the numbering here starts from zero, then in order to take the second line, we indicate 1

Next, you need to specify the group number and the variable to which the result will be saved. Here, too, the numbering of groups starts from zero. But group 0 contains the entire found line (11.12.2013, 01:22:55.3021). Therefore, for the day we indicate the group number 1, for the month - 2 and for the year - 3.

To table

It is very similar to the previous function (To variables) with the difference that not one result is saved here, but everything is stored in a table. You can exclude some of the groups found from the final result.

Click here to expand the example

We use the same text:

21.01.2003, 11:34:00.9299 11.12.2013, 01:22:55.3021 04.01.2007, 08:00:06.0032

Our task is to parse it and save it to a table. To do this, we will use the following regular expression: (\d{2}).(\d{2}).(\d{4}), (\d{2}):(\d{2}):(\d{2}).(\d{4})

This is how the output looks in the Regular Expression Tester:

Open

Let's also imagine that we don't need seconds and milliseconds in the final table. This is how it might look:

Open

The group under index 0 contains the entire match (in our case, the string), so we exclude it. In groups 6, 7 - seconds and milliseconds, respectively.

Usage example

Let's look at a specific example - parsing links with regular expressions, composed using a tester.

For example, we have a task - to get links to the profiles of active users of the ZennoLab forum. Let's get started:

Open

1. With the help of the cube Get value, we get the HTML code of the element in which links to users who are online on the forum are posted.

2. Add the “Regex” action. To compose the pattern used in the properties of the “Regex” action, use the Regular Expression Tester.

3. Add the “html“ variable to the input in the action properties, and save the result to the “urls” list.

4. After running the cube, we get unique id in the list, which can be used to form the URL of user profiles.

Spintax

Randomization or uniqueness of text. With the help of spintax it is convenient to create synonymization of texts. Spinax is a construction of curly braces and vertical slashes that allows you to randomly substitute substrings from a string. In its simplest form, the spintax looks like this: {variant1 | variant2 | variant3}. When executing this action, one of the three options will accidentally fall into the resulting variable.

But spintax constructions can be more complex and have multi-level nesting, which is why you can get thousands of different variants from one text.

Open

Extended spintax syntax

• {Red|White|Blue} - the resulting text contains one of the values, for example: "White"

• [ Red| White| Blue] - the resulting text contains a permutation of values, for example: "White Blue Red"

• [+_+Red|White|Blue] - the resulting text contains a permutation of values between which a separator is inserted, for example: "White_Red_Blue"

Nesting of templates is unlimited (for example: [+{_|-}+Red|White|Blue {1|2}] = «White-Blue 2-Red»). Special characters can be escaped:[+\++Red|\[White\]|Blue]-result «[White]+Red+Blue»

Split

Separation of text with any separator character (delimeter). This processing turns the string into an array of strings. In fact, this is a simpler analogue of RegExp for separating a string with characters.

Separators

Here you need to specify the symbol (s) by which the data will be split.

Open

Allow empty values

Let's look at this point with an example.

And so we have a string in the format name;surname;gender;year of birth An action might look like this:

Open

But, if one of the components is missing, for example gender (Andrew;Paul;;1988) , then the year of birth will be written to the variable for gender (sex) This is exactly what the Allow empty values setting has been created for such cases - if you enable it, an empty string will be written to the gender variable, and the year will be saved to the correct variable.

Usage example

Let's consider the work of a split using an example of a very common task - splitting a string with a proxy into its constituent parts. Very often purchased proxies have the following format: login:pass@host:port

There are two separators at once -: (colon) and @. This is what the action settings might look like:

Open

Both characters are indicated here as a separator.

ToChar

Converts an integer value to Unicode characters .
Each Unicode character has its own numeric code and this functionality allows you to convert a numeric value to the corresponding characters. For example, the symbol ♛ has a numeric value 9819

Open

ToLower

Changes letters to lowercase depending on the selected property: either all letters, or only the first letter of the string, or the first letter in each word.

For example, we are VERY ANNOYED BY WRITING CAPS LOCK . We pass the text through this macro and get the normal spelling.

Open

Changes the case of letters to lowercase depending on the selected property. For example, let's take the line

CHANGES REGISTER OF LETTERS TO LOWER

All

Replaces all uppercase characters in the text with lowercase.

Beginning of words

Changes the case to lowercase for the first character of each word in the text.

The first character

Changes the case of only the first character in the specified text.

ToUpper

Open

Changes the case of letters to uppercase depending on the selected property. For example, let's take a string lowercase text

All

Replaces all lowercase characters in the text with uppercase.

Beginning of words

Changes the case of the first character of each word in the text to uppercase.

The first character

Changes the case of only the first character in the specified text.

Trim

This function is used to remove extra characters at the beginning and \ or end of the passed string.

It is most often used when you need to clean up a string from extra spaces, line breaks, tabs, which so often remain as a result of parsing.

What to cut

Here you need to select the characters to be deleted. It can be either a preset option for all types of whitespace characters (space, enter, tabulation), or you can specify your own characters.

Where to trim

Where to trim characters - Beginning, End, or Beginning and End.

UrlDecode

Decodes an UrlEncode encoded string (described below).

Was: Hello+G%c3%bcnter!
It became: Hello Günter!

Open

UrlEncode

Only Latin letters, numbers and a few punctuation marks are allowed in the URL. All other characters that are transmitted in the HTTP request must be encoded using UrlEncode, otherwise the server may misinterpret the request.

Encode only variables values

It is very convenient to use when making HTTP requests, because you do not need to encode the site address, but only the parameters. This is what the action settings might look like:

Open

The {-Variable.keyword-} variable contains the text what urlencode is. After execution, the following line will be written to the {-Variable.url-} variable -https://www.google.com/search?q=%d1%87%d1%82%d0%be+%d1%82%d0%b0%d0%ba%d0%be%d0%b5+urlencode

To variable

This action simply saves everything that you add to the input window - variables, text, symbols, project constants, into a separate variable.

Open

To list

This action splits the text using the specified in the properties of the separator into lines and writes them to the list.

Separator

• Enter - newline character

• Space

• Custom text - here you can specify both a single character (for example;) and several characters (note: if you specify several characters here, they will be considered as one separator!)

• Custom Regex - using regular expression .

Open

To table

This action splits the transmitted text into rows and columns (according to the specified separators) and places the data into a table.

Separators

• Enter - newline character

• Space

• Custom text - here you can specify both a single character (for example;) and several characters (note: if you specify several characters here, they will be considered as one separator!)

• Custom Regex - using regular expression.

Replace

Open

This action searches the string for a substring, replaces it with another, and then saves the result to a variable.

What to search

The substring to find (or a regular expression if the Regex search type is selected).

What to replace

What the found substring will be replaced with.

Search type

Text - exactly the same string that was passed in the What to search field is searched for.

Regex - in the What to search field, write a regular expression that will be used to search for a match.

What to replace

First

The first match found will be replaced.

All

All matches will be replaced.

One match

Replace only one specified match (numbering from zero!) Or Last.

Matches No.

Specify, separated by commas, the numbers of the matches to be replaced (numbering from zero!).

Translation

Translates strings from one language to another.

Open

The translation action has a large selection of translation services, which will help you flexibly approach the uniqueization of texts by choosing the highest quality texts.

Translation service

Here you need to select the service through which the transfer will be carried out. Available options:

• Baidu

• DeepL

• Google

• Google via web interface

• Microsoft

• Yandex

API keys for services can be added in ZennoPoster settings.

Source language, Target language

From which and into which language the text should be translated.

Here are some examples: English - en, Spanish - es, German - de, Russian - ru (full list)

Additional parameters

Additional parameters that can be passed should be clarified in the documentation of the selected service.

Use project proxies (if possible)

If possible, the transfer request will be made using the currently installed proxy.

Prepare JavaScript

Processes a string for correct use in JavaScript. Mostly escapes quotes and other specials. symbols. This macro prepares text so that it can be inserted as a string in a JavaScript or IF action. ProjectMaker has a JavaScript tester where you can inspect (test) your code. This "cube" will help you to escape quotes, apostrophes and other special characters.

It was: <a href="https://zennolab.com/">
Now : <a href=\"https://zennolab.com/\">

Substring

Takes a piece of text from a string specified in the action properties by two indices - from one character to another. For example, if you take the first sentence of this paragraph and there is a task to get a substring in it from 106 characters to the end of the text, then we get “to another.”.

Open

Transliteration

Sometimes it is required to write Cyrillic words using Latin alphabet. You can do it with this action.

Open

Useful links

• Regular Expression Tester

• Variable window