Text processing

Owned by Stanislav N. (Unlicensed)
Last updated: Sept 11, 2021 by ZL Support
12 min read

Table of contents

Description

This block serves various manipulations with text, which are very often required in practice. Process the parsed text, clean it of garbage, translate it into other languages - all this, and much more, can be done by a text processing "cube".

How to add an action to a project?

Via context menu Add Action -> Data -> Text Processing

Or use smart search.

Where is word processing applied?

How to use the action?

The properties window consists mainly of three areas:

  1. The input string is text, a variable, or a combination of both.

  2. Actions on the string, properties and their settings.

  3. The output string (result) in a variable.

Set the cursor in the input line area, press Ctrl + Space and select useful constants and project variables from the drop-down list. For example, this way you can quickly insert a proxy of the project {-Project.Proxy-} or the URL of the active tab {-Page.Url-} (you can find other available environment variables in the article Variables window)

All possible operations with this "cube":

Escape sequences

Escaping sequences. This action replaces the characters * +? | {[() ^ $. # (places a "slash" in front of each specified character -\ ) This technique is often used to work with queries and for the regex engine to use these characters literally, rather than as commands or metacharacters.

Before Application: {"animal": "cat"}
After: \ {"animal": \ "cat"}

Regex

Processing text with regular expressions.
Regulars are very convenient to parse strings to find the required substring for a given pattern. This action allows you to parse not only the first found value, but the entire group and save the values to variables or a table.

Input field “Regex”

In this field, you must enter a regular expression, which will be used to search the text. Example -(?<=<title>).*(?=</title>)

Regular Expression Tester can help you in writing regular expressions

Error with an empty answer

If this setting is checked and the regular expression does not find anything in the text, then the action will fail (exit via the red branch).

Please note that if the regular expression returns an empty string, then even if the "Error with empty response" setting is enabled, the action will be launched on the green branch: for example, the site has nothing in the title tag: <title></title>, in this In case the regular expression (?<=<title>).*(?=</title>) will work, but return an empty string - the action will succeed.
But if there were no <title></title> in the text at all, in this case the expression will not find anything and the action will be launched on the red thread.

What to get

First

The first match found will be saved to the variable.

All

Save all search results to a list.

One match

Keep only one match.
In the field that appears, you can enter the sequence number of the match (numbering from zero!) Or select the Last or Random (random) value.

Match No.

Save to the list only the specified match numbers (numbering from zero !, specify separated by commas).

To variables

This function is used when working with group regular expressions. An example under the spoiler:

Let's imagine that there is the following text:

21.01.2003, 11:34:00.9299 11.12.2013, 01:22:55.3021 04.01.2007, 08:00:06.0032

And the task is to disassemble it into its components. To do this, we will use the following regular expression: (\d{2}).(\d{2}).(\d{4}), (\d{2}):(\d{2}):(\d{2}).(\d{4})

This is how the output looks in the Regular Expression Tester:

Let's imagine that we need to take in variables the day, month and year from the second row. Here's how you can do it:

The match number in our case is the line number. Because the numbering here starts from zero, then in order to take the second line, we indicate 1

Next, you need to specify the group number and the variable to which the result will be saved. Here, too, the numbering of groups starts from zero. But group 0 contains the entire found line (11.12.2013, 01:22:55.3021). Therefore, for the day we indicate the group number 1, for the month - 2 and for the year - 3.

To table

It is very similar to the previous function (To variables) with the difference that not one result is saved here, but everything is stored in a table. You can exclude some of the groups found from the final result.

We use the same text:

21.01.2003, 11:34:00.9299 11.12.2013, 01:22:55.3021 04.01.2007, 08:00:06.0032

Our task is to parse it and save it to a table. To do this, we will use the following regular expression: (\d{2}).(\d{2}).(\d{4}), (\d{2}):(\d{2}):(\d{2}).(\d{4})

This is how the output looks in the Regular Expression Tester:

Let's also imagine that we don't need seconds and milliseconds in the final table. This is how it might look:

The group under index 0 contains the entire match (in our case, the string), so we exclude it. In groups 6, 7 - seconds and milliseconds, respectively.

Usage example

Let's look at a specific example - parsing links with regular expressions, composed using a tester.

For example, we have a task - to get links to the profiles of active users of the ZennoLab forum. Let's get started:

  1. With the help of the cube Get value, we get the HTML code of the element in which links to users who are online on the forum are posted.

  2. Add the “Regex” action. To compose the pattern used in the properties of the “Regex” action, use the Regular Expression Tester.

  3. Add the “html“ variable to the input in the action properties, and save the result to the “urls” list.

  4. After running the cube, we get unique id in the list, which can be used to form the URL of user profiles.

Spintax

Randomization or uniqueness of text. With the help of spintax it is convenient to create synonymization of texts. Spinax is a construction of curly braces and vertical slashes that allows you to randomly substitute substrings from a string. In its simplest form, the spintax looks like this: {variant1 | variant2 | variant3}. When executing this action, one of the three options will accidentally fall into the resulting variable.

But spintax constructions can be more complex and have multi-level nesting, which is why you can get thousands of different variants from one text.

Extended spintax syntax

  • {Red|White|Blue} - the resulting text contains one of the values, for example: "White"

  • [ Red| White| Blue] - the resulting text contains a permutation of values, for example: "White Blue Red"

  • [+_+Red|White|Blue] - the resulting text contains a permutation of values between which a separator is inserted, for example: "White_Red_Blue"

Nesting of templates is unlimited (for example: [+{_|-}+Red|White|Blue {1|2}] = «White-Blue 2-Red»). Special characters can be escaped:[+\++Red|\[White\]|Blue]-result «[White]+Red+Blue»

Split

Separation of text with any separator character (delimeter). This processing turns the string into an array of strings. In fact, this is a simpler analogue of RegExp for separating a string with characters.

Separators

Here you need to specify the symbol (s) by which the data will be split.

Allow empty values

Let's look at this point with an example.

And so we have a string in the format name;surname;gender;year of birth An action might look like this:

But, if one of the components is missing, for example gender (Andrew;Paul;;1988) , then the year of birth will be written to the variable for gender (sex) This is exactly what the Allow empty values setting has been created for such cases - if you enable it, an empty string will be written to the gender variable, and the year will be saved to the correct variable.

Usage example

Let's consider the work of a split using an example of a very common task - splitting a string with a proxy into its constituent parts. Very often purchased proxies have the following format: login:pass@host:port

There are two separators at once -: (colon) and @. This is what the action settings might look like:

Both characters are indicated here as a separator.

ToChar

Converts an integer value to Unicode characters .
Each Unicode character has its own numeric code and this functionality allows you to convert a numeric value to the corresponding characters. For example, the symbol ♛ has a numeric value 9819

ToLower

Changes letters to lowercase depending on the selected property: either all letters, or only the first letter of the string, or the first letter in each word.

For example, we are VERY ANNOYED BY WRITING CAPS LOCK . We pass the text through this macro and get the normal spelling.

Changes the case of letters to lowercase depending on the selected property. For example, let's take the line

CHANGES REGISTER OF LETTERS TO LOWER

All

Replaces all uppercase characters in the text with lowercase.

It was

Has become

It was

Has become

CHANGES REGISTER OF LETTERS TO LOWER

changes the case of letters to lowercase

Beginning of words

Changes the case to lowercase for the first character of each word in the text.

It was

Has become

It was

Has become

CHANGES REGISTER OF LETTERS TO LOWER

cHANGES rEGISTER oF lETTERS tO lOWER

The first character

Changes the case of only the first character in the specified text.

It was

Has become

It was

Has become

CHANGES REGISTER OF LETTERS TO LOWER

cHANGES REGISTER OF LETTERS TO LOWER

ToUpper

Changes the case of letters to uppercase depending on the selected property. For example, let's take a string lowercase text

All

Replaces all lowercase characters in the text with uppercase.

It was

Has become

It was

Has become

lowercase text

LOWER CASE TEXT

Beginning of words

Changes the case of the first character of each word in the text to uppercase.

It was

Has become

It was

Has become

lowercase text

Lower Case Text

The first character

Changes the case of only the first character in the specified text.

It was

Has become

It was

Has become

lowercase text

lower case text

Trim

This function is used to remove extra characters at the beginning and \ or end of the passed string.

It is most often used when you need to clean up a string from extra spaces, line breaks, tabs, which so often remain as a result of parsing.

What to cut

Here you need to select the characters to be deleted. It can be either a preset option for all types of whitespace characters (space, enter, tabulation), or you can specify your own characters.

Where to trim

Where to trim characters - Beginning, End, or Beginning and End.

UrlDecode

Decodes an UrlEncode encoded string (described below).

Was: Hello+G%c3%bcnter!
It became: Hello Günter!

UrlEncode

Only Latin letters, numbers and a few punctuation marks are allowed in the URL. All other characters that are transmitted in the HTTP request must be encoded using UrlEncode, otherwise the server may misinterpret the request.

Encode only variables values

It is very convenient to use when making HTTP requests, because you do not need to encode the site address, but only the parameters. This is what the action settings might look like:

The {-Variable.keyword-} variable contains the text what urlencode is. After execution, the following line will be written to the {-Variable.url-} variable -https://www.google.com/search?q=%d1%87%d1%82%d0%be+%d1%82%d0%b0%d0%ba%d0%be%d0%b5+urlencode

To variable

This action simply saves everything that you add to the input window - variables, text, symbols, project constants, into a separate variable.

To list

This action splits the text using the specified in the properties of the separator into lines and writes them to the list.

Separator

  • Enter - newline character

  • Space

  • Custom text - here you can specify both a single character (for example;) and several characters (note: if you specify several characters here, they will be considered as one separator!)

  • Custom Regex - using regular expression .

To table

This action splits the transmitted text into rows and columns (according to the specified separators) and places the data into a table.

Separators

  • Enter - newline character

  • Space

  • Custom text - here you can specify both a single character (for example;) and several characters (note: if you specify several characters here, they will be considered as one separator!)

  • Custom Regex - using regular expression.

 

Replace

This action searches the string for a substring, replaces it with another, and then saves the result to a variable.

The substring to find (or a regular expression if the Regex search type is selected).

What to replace

What the found substring will be replaced with.

Search type

Text - exactly the same string that was passed in the What to search field is searched for.

Regex - in the What to search field, write a regular expression that will be used to search for a match.

What to replace

First

The first match found will be replaced.

All

All matches will be replaced.

One match

Replace only one specified match (numbering from zero!) Or Last.

Matches No.

Specify, separated by commas, the numbers of the matches to be replaced (numbering from zero!).

Translation

Translates strings from one language to another.

The translation action has a large selection of translation services, which will help you flexibly approach the uniqueization of texts by choosing the highest quality texts.

Translation service

Here you need to select the service through which the transfer will be carried out. Available options:

API keys for services can be added in ZennoPoster settings.

Source language, Target language

From which and into which language the text should be translated.

Here are some examples: English - en, Spanish - es, German - de, Russian - ru (full list)

Additional parameters

Additional parameters that can be passed should be clarified in the documentation of the selected service.

Use project proxies (if possible)

If possible, the transfer request will be made using the currently installed proxy.

Prepare JavaScript

Processes a string for correct use in JavaScript. Mostly escapes quotes and other specials. symbols. This macro prepares text so that it can be inserted as a string in a JavaScript or IF action. ProjectMaker has a JavaScript tester where you can inspect (test) your code. This "cube" will help you to escape quotes, apostrophes and other special characters.

It was: <a href="https://zennolab.com/">
Now : <a href=\"https://zennolab.com/\">

Substring

Takes a piece of text from a string specified in the action properties by two indices - from one character to another. For example, if you take the first sentence of this paragraph and there is a task to get a substring in it from 106 characters to the end of the text, then we get “to another.”.

Transliteration

Sometimes it is required to write Cyrillic words using Latin alphabet. You can do it with this action.