Task scheduling

Use task scheduling to automate the execution of jobs to perform regularly or at a defined time and date.

Define the automated actions for saving reports, sending reports by e-mail, moving data to an FTP site, and executing Jython scripts or batch files in a Windows shell. A task represents one or more commands (and their parameters) launched at a given point in time, and repeated after a given time period.

Task execution is controlled by a scheduler module that controls the task start and finish and the task recovery manager which ensures the execution of the tasks that have been interrupted by a system failure.

Tasks are defined in the XML files that you place in the config directory on the disk where your report server is installed. Provided your NAM Server was installed in the default location, the full path to this directory is:

<installation directory>\config\

Tasks are defined in the XML files that you place in the report server config directory (<installation directory>\config\).

The task scheduler configuration files are named according to the following naming convention:

tasks-nnn -*.xml

Where nnn represents a three-digit number and the asterisk (*) is a wildcard that represents any string that is valid in a file name.

All of these files are executed by the task scheduler. The number embedded in the file name determines the order in which the files are read by the scheduler. The directory contains several pre-defined scheduler files, such as config\tasks-300-iti.xml. You can use an existing file as a template to create your own configuration file, or you can edit an existing configuration file. The latter is not recommended, however, because the pre-defined configuration files are overwritten during a system upgrade.

Task scheduling overview

The scheduled tasks on the report servers are stored in XML files that contain a sequence of one or more commands. Each task definition consists of, for example, a task ID, name, time-related properties (start time, period and timeout), and the task type definition. Tasks can be launched automatically by the scheduler or manually by the user.

By default, tasks are executed sequentially, but it is possible to launch a task in a new thread (see the attribute threadFromCmd) and continue launching the other tasks.

Task definitions can update or modify the original definition, even from another file. In this situation, several tasks share the same ID - one original (with mergeMode=NONE) and many updating (with mergeMode<>NONE). When the task is loaded, these definitions are merged into one task in accordance with their mergeMode (see the attribute mergeMode).

Data timeline tasks

Based on their launch factor, tasks can be divided into two categories: server tasks and data timeline tasks. This means that task execution may be dependent on the time or on data arrival.

The launch of a data timeline task is triggered by the arrival of new data. The task is executed when new data arrives and if its next run is not scheduled later than the data time stamp. The task execution may be delayed if the previous task instance or another non-threaded task is still running. Then the queued task waits for the termination or timeout of the active task.

For tasks with the ID set to Daily (ID="Daily"), to avoid data congestion, you can use the property SCHED_SAMPLES_SUMMARY_SKIP_FACTOR defined in the common.properties file (by default, the property is not set). The value of this property is used as a multiplier of the task period and is included in calculations when the scheduled task contains the summary type commands (baselines) and a portion of data still awaits processing. In these situations, the queued task is skipped when the time interval between the last data and the current processed data time stamps is greater than the queued task's period multiplied by SCHED_SAMPLES_SUMMARY_SKIP_FACTOR.

Server tasks

The Server task launch is triggered by the server time alone. A task is executed if its next run time stamp is not scheduled later than the server time. The task execution may be delayed if the previous task instance or another non-threaded task is still running. In these situations, the queued task waits for termination or timeout of the active task. In any case, the task receives nextRun as a time stamp.

Task recovery

After a server failure, the scheduler re-executes the interrupted recoverable tasks (marked in the definition file as recoverable). The task execution during recovery starts from the command that was interrupted. Recovery runs just after the scheduler start, before normal scheduled tasks start running. If the second recovery attempt fails, the command will not be recovered in the next turn.

Task parameters

A task has the following time-related run parameters:

offset

The time of the first task execution.

The task execution starts at offset time and is repeated every period until the server stops. If time period unit is lesser than DAY, the offset is followed only the first time the task is run. To execute two or more tasks at the same moment (if two tasks have identical offset settings), they are run in the order in which they were defined.

period

The period of successive task executions.

timeout

The maximum allowed time of task duration. If a task does not end after the timeout is reached, it is treated as a failed task.

timestamp

The time of the current task execution.

nextRun

The time stamp of the next task launch.

  • Server tasks are launched when the current time is later than the task's nextRun.
  • Data timeline tasks are launched when the time stamp of the arriving data is later than task's nextRun.

The nextRun is calculated after each task launch using the previous values of the nextRun, offset, and period parameters.

Task attributes

When you define a task, each <task> element must contain attributes whose values will determine, at a minimum, the task's unique identifier, type (server or data timeline task), and recurrence factor.

Other possible attributes control how a task behaves when other tasks try to merge with it, when the task starts, how long it should last, whether it should be recovered after server failure, and whether it should be placed in a sequence (as a non-threaded task) or started immediately (as a threaded task).

The available task attributes are:

ID

Unique task identifier. If multiple task definitions have the same ID, they are merged into one task.

Type: string

Importance: required

name

The task name displayed in configuration servlets.

Type: string

Importance: optional

Default value: empty

mergeMode

Defines how the task will merge with its originator. In some cases, you can have an incomplete task definition. In other cases, all required elements must be defined. The valid values are:

NONE

The task is original. Task definition must be complete.

JOIN

The existing parameters are overwritten, non-existing are added, commands with the same ID are overwritten, and other commands are appended. The task definition can be incomplete.

REPLACE

The updating task parameters and commands completely replace original task contents. Task definition must be complete.

REMOVE

The task with this ID is removed from scheduler completely.

Type: string

Importance: optional

Default value: NONE

timeLine

Determines the timeline considered during the task execution. The valid values are:

SERVER

The server timeline task. The task is first executed at offset time and then repeated after every interval.

SAMPLES

The data timeline task. The task is first executed at offset time and then repeated after every interval, but only if new data arrives after this time.

Type: string

Importance: required

periodType

How often the task will be executed. The valid values are:

ALWAYS

The task is executed in every launch sequence (server timeline task) and upon arrival of every data portion (data timeline task).

MINUTE

The task is first executed at the offset time (hh:mm counted from midnight) and then repeated after every n minutes.

HOUR

The task is first executed at the offset time (hh:mm counted from midnight) and then repeated after every n hours at the offset minutes.

DAY

The task is first executed at the offset time (hh:mm counted from midnight) and then repeated after every n days at the offset time.

WEEK

The task is first executed at the offset time (dd:hh:mm counted from Sunday midnight, dd is the index of day in week, counted from 0) and then repeated after every n weeks at the offset day and time.

MONTH

The task is first executed at the offset time (dd:hh:mm counted from midnight of first day in month, dd is index of day in month, counted from 0) and then repeated after every n months at the offset day and time.

ONCE

The task is executed only once, after the server start (server timeline task) or after arrival of the first data portion (data timeline task). If the task definition changes, the task will be executed once again after its definition is automatically reloaded.

Type: string

Importance: required

period

How often the task will be executed in periodType (see periodType) units.

Type: long

Importance: required

offsetTime

The task execution offset, counted from midnight. The value can contain days as first field (only for periodType WEEK (max. 7 days) or MONTH (max. 31 days)).

Type: string ([dd:]hh:mm)

Importance: optional

Default value: 00:00:00

timeout

The maximum expected execution time. After this time, the task is considered to be failed.

Type: string (hh:mm)

Importance: optional

Default value: 10 minutes (00:10)

recoverable

Determines whether the task should be recovered after system failure. The task recovery manager picks only tasks and their commands that were active during system failure. During recovery, all commands following the interrupted one are also executed.

Type: string

Importance: optional

Default value: false

threadFromCmd

The ID of the task command triggering execution of the whole task in new thread. When the task is executed in a new thread, the scheduler does not wait for task completion and executes the next task immediately. The possible values are:

No attribute

The task cannot be executed in new thread.

Empty string

The task should be executed in new thread. If the string is empty, insert the attribute in the task but do not assign a value to it.

 threadFromCmd=""

Command ID

Execute the task in a new thread originating from the command with a given ID.

Type: string

Importance: optional

Executing server tasks automatically

With normal, undisturbed operation, the scheduler on the report server launches tasks automatically in the order in which they were loaded. The tasks run according to the time-related parameters.

The tasks are launched automatically:

  • At the server start, the scheduler loads task definitions from XML files and then computes nextRun for each server task. For the data timeline tasks, the calculations are done after the arrival of the first portion of data. The base for the calculation is taken either from the last launch time stamp stored in the file <installation directory>\config\scheduled\TaskScheduler.state or from the current server time if the TaskScheduler.state file does not exist.
  • If a previous instance of the task is still running, the scheduler waits for its termination or timeout and then launches this task and continues with the subsequent tasks.
  • After the task launch, the scheduler sleeps until the beginning of the next minute and then starts the task launch sequence again.
  • After the task recovery is complete, the scheduler starts launching the server tasks. Tasks with a nextRun less than the current time are launched once, in ascending order of nextRun, with the nextRun value set as timestamp.
  • After each task execution, the scheduler calculates the value of the nextRun property for the task. If this value is less than the current time, the task skipped some of its scheduled launches. When this occurs, only the last nextRun before the current time is taken and the task is executed one more time with this value.
  • The data timeline tasks are launched in the same way as the server tasks, but the launch sequence is triggered by the data arrival and the data time stamp, not the current time.
  • Depending on the value of the periodType attribute (see periodType), a task's recurrence can be set to:
    • ALWAYS: the task is executed at every scheduler attempt: a server task is executed in each launching sequence; a data timeline task is executed after the arrival of each data portion.
    • ONCE: the task is executed only once, after server start or after the arrival of the first data portion. If the task definition has changed, the task is executed again after the task definitions are reloaded automatically.

Executing server tasks manually

You can execute any server or data timeline task manually. You determine the time stamp.

Before you schedule server tasks, you should already be familiar with task concepts. See Task scheduling overview.

When launching tasks manually, you must choose tasks that have commands to execute and a time interval within which they will be launched. For each task chosen for manual execution, the scheduler computes a sequence of time stamps in the given interval, according to task's offset and period, and clones the task so that there is a copy of the task for each calculated time stamp. As a result, the task will be launched as multiple instances, each with a different time stamp.

Note
  • Adding a new manual task clears the previous list.
  • Manual tasks cannot be added while other manual tasks are still running.
  • After a manual start, all tasks are executed in the order of their time stamps. Any task with a running normal instance (a data timeline task or server task) will be skipped.
  • If a task contains summary-type commands (baselines), they will be launched only in the last instance of the interval.
  • When the attribute periodType (see periodType) is ALWAYS or ONCE, only one instance is created regardless of the time interval you define.

To run a server task manually:

  1. On the NAM Server, select Tools ► Admin console.
  2. In the System management column, click Task schedule status.
  3. Click the name of the command you want to run.
  4. In the pop-up window, select the commands to run and then click Close.
    There is now a check in the Execute column.
  5. Set the Begin date and End date for task completion. By default, a one day interval will be set, the End date being the current server time. Follow the year-month-day hour:minute time format as displayed in brackets next to each field. For example, 09-04-23 06:33 .
  6. At the bottom of the screen, click Start manual tasks.

Command classes and attributes

A command is a single action within a scheduled task. When creating tasks and defining commands to be executed, you must assign the command, its class, and all attributes.

Command tags

class

The qualified name of the class executed in command (extension of adlex.delta.scheduler.command.Command). Example: adlex.delta.scheduler.command.SaveReportCommand .

Type: string

Importance: required

param

The class launch parameter. A command can have many parameters; each must be a separate tag. Command parameters can contain special tags. Note that tasks are stored as XML files and some special characters must be represented by entities so that they are not misinterpreted by the XML parser. The valid parameters are:

[timestamp]

For the data timeline tasks, this parameter represents the time stamp of the data portion. For the server tasks, this parameter represents the time stamp of the current or scheduled task run.

{file.separator}

Represents the file separator in a file or directory path.

{delta.root}

Represents the path to the root directory of the report server.

&amp;

Represents the “&” character.

&lt;

Represents the “<” character.

&gt;

Represents the “>”character.

Type: string

Importance: optional

Command attributes

ID

The unique command identifier (unique within a task).

Type: string

Importance: required

name

The command name displayed in configuration servlets.

Type: string

Importance: optional

Default value: empty

timeout

The maximum expected execution time. After this time, the command is regarded as failed.

Type: string (hh:mm, such as 5:45)

Importance: optional

Default value: inherited from the command's task timeout attribute.

Commands and parameters

Each command, depending on its class, can have a number of required or optional parameters that act as additional instructions to the server when it is scheduling and launching jobs.

SaveReportCommand

The SaveReportCommand is a command for storing report results on a server disk without using an HTTP request.

class

adlex.delta.scheduler.command.SaveReportCommand

param

The relative URL of a saved report (server address must omitted). For example:

/LSSaveAs?lsAction=Saveas&amp;format=pdf&amp;lsEntryName=Test

URL parameters:

  • format=[pdf] : The format of report file.
  • lsEntryName : The name of report to save.
  • lsEntryId : The ID of report to save.

param

The report file name and path on the server for saving with the report results. For example:

{delta.root}{file.separator}Reports{file.separator}saved_report.pdf

NO_FILE - The report result will not be saved on disk.

ReportCommand

The ReportCommand is a command for executing a report via HTTP and storing the results on a server disk. Use the SaveReportCommand to save a report only. The ReportCommand is suitable for performing certain tasks regularly on another server or to control the behavior of several report servers.

class

adlex.delta.scheduler.command.ReportCommand

param

The URL of the saved report.

Note that (the “»” character at the end of a line indicates that the statement continues on the next line:

http://localhost/ »

LSServlet?lsAction=Saveas&amp;format=pdf&amp;lsEntryName=Test

param

The report file name and path on the server for saving the report results. For example:

{delta.root}{file.separator}Reports{file.separator}saved_report.pdf

NO_FILE - The report results will not be saved on disk.

FTPCommand

FTPCommand is used for transferring files through FTP. The files are uploaded or downloaded in binary mode.

class

adlex.delta.scheduler.command.FTPCommand

param

FTP commands:

  • PUT
  • GET

param

The server credentials. For example: user:password@server:port

param

The source file path on a local or remote server. For example:

  • local path: {delta.root}{file.separator}reports{file.separator}foo_report.txt
  • remote path on FTP server:
    upload/reports/saved_report.txt

param

The destination directory path. For example:

upload/reports or {delta.root}{file.separator}reports

SMTPCommand

The SMTPCommand is used to send files by email.

class

adlex.delta.scheduler.command.SMTPCommand

param

The path on the server of the file to send. For example:

{delta.root}{file.separator}Reports{file.separator}saved_report.pdf

param

The list of email addresses (comma separated). For example:

john.smith@mail.com,ann.brown@mail.com

param

The email subject. For example: Daily Report

param

The MIME type of the email. For example: text/plain .

SMTPReportCommand

The SMTPReportCommand is a command used for executing a report via HTTP and sending the results by e-mail. The order of the parameters is important. Begin with the report URL, then add a recipient's address, and finish the definition with the email subject.

class

adlex.delta.scheduler.command.SMTPReportCommand

param

The path part of a URL used to access the report, that is the part right after the server's address, including the first slash / character. Make sure to encode the ampersand &amp; character with &amp;.

/LSServlet?lsAction=Load&amp;lsEntryId=575&amp;dmiAction=Generate

param

The list of email addresses (comma separated). For example:

john.smith@mail.com,ann.brown@mail.com

param

The email subject. For example: Application Performance.

SQLQueryExportCommand

The SQLQueryExportCommand is used for executing the SQL queries specified in an external file and exporting the results to a text file.

class

adlex.delta.scheduler.command.SQLQueryExportCommand

param

The time stamp in milliseconds representing the start time of the task. It is usually set to the [timestamp], indicating that it is replaced by the task scheduler.

param

Path to a file containing the SQL query to be executed.

In queries you can use the phrase [nnn], where nnn is the number of daily tables to be processed. You can use [timestamp] to insert a time stamp value. When you use, for example, select * from rtmnetstats[97]where samp_time=[timestamp] in the file containing the query, the phrase [97] is replaced with the right table based on the time stamp parameter and [timestamp] is replaced by data or task time stamp (depending on the task type).

Other possible variants include:

  • select * from rtmnetstats0
  • select * from rtmnetstats1 or select * from rtmnetstats97

param

The path to a file where the results of the query will be exported. For example:

{delta.root}{file.separator}Reports{file.separator}?exported_result.txt

NONE – The report result will not be saved on disk.

param

The maximum number of rows to be processed. For example, 30.

MAX – Unlimited number of rows.

param

The timeout – after the specified number of seconds, the query will be terminated (obsolete, command timeout is used instead).

Leave this field empty.

param

This is an optional parameter. Set to APPEND to append the query result to an existing output file. By default the output file is overwritten.

param

This is an optional parameter. Column delimiter. If not specified, a comma is used.

param

This is an optional parameter. Row delimiter. If not specified, a newline character “\n ” is used.

Jython

The Jython command is used for executing the Jython script on the server.

class

adlex.delta.scheduler.command.Jython

param

The Jython script file path. For example:

{delta.root}{file.separator}Jython{file.separator}script.py

param

The output file path. For example:

{delta.root}{file.separator}Jython{file.separator}file.out

ShellCommand

The ShellCommand is used for executing the shell commands on the server.

Note

A shell command or program must never try to write to standard output or read from standard input. You must redirect the output to a file or set program parameters to force no output text (quiet mode). For example, always use the /q parameter for cmd.exe .

class

adlex.delta.scheduler.command.ShellCommand

param

The shell command processor (cmd.exe for Windows).

param

The shell command processor parameters. For example: /q /c.

param

The command executed in shell. For example: copy.

param

The parameters of the command executed in shell. These can be given in native server format (including OS-specific file separators or absolute paths). For example:

C:\DIR\FILE.txt

Task definition example

Each scheduled task must contain at least one command (class extending adlex.delta.scheduler.command.Command) that defines an action to take. The commands are executed in order by ID.

Sample task file

The following task definition contains the commands for exporting a DMI report to a PDF file, uploading it to an FTP site, and sending it by email. The name of the file that contains the task rules is tasks-9999-example.xml . Note that command IDs are numbers, ensuring that they are executed in the required order, because the commands are executed in order by ID.

<tasks>

<task ID="ExportReport" name="Export report to a file" periodType="DAY" period="1" timeLine="SERVER" offsetTime="15:20" timeout="00:02">  

<command ID="0" name="Report generation and export">  
    <class>adlex.delta.scheduler.command.SaveReportCommand</class>
        <param>/LSSaveAs?lsAction=Saveas&amp;format=pdf&amp;lsEntryName=Heavy+Usage+Subscribers</param>
        <param>{delta.root}{file.separator}temp{file.separator}Heavy_Usage_Subscribers.pdf</param>
</command>

<command ID="1" name="FTP upload">  
    <class>adlex.delta.scheduler.command.FTPCommand</class>
        <param>PUT</param>
        <param>user:password@IP_ADDR</param>
        <param>{delta.root}{file.separator}temp{file.separator}Heavy_Usage_Subscribers.pdf</param>
        <param>/pub/upload/vsc/export</param>
</command>

<command ID="2" name="Report sending">  
    <class>adlex.delta.scheduler.command.SMTPCommand</class>
        <param>{delta.root}{file.separator}temp{file.separator}Heavy_Usage_Subscribers.pdf</param>
        <param>odbiorca@plus.pl</param>
        <param>Heavy Usage Subscribers report</param>
        <param>pdf/pdf</param>
</command>

</task>

</tasks>

Callout number 1 A definition of a daily server task (executed once a day) named “Export report to a file”, starting at 15:20 (3:20 PM) and lasting for no more than 2 minutes.

Callout number 2 Report generation and export to a file.
This command saves a given report to a PDF file. The DMI report is referred to by its name. In this example, the report exported is called “Heavy Usage Subscribers” and is accessed in a web browser via the /LSServlet?lsAction=Load&lsEntryId=118&dmiAction=Generate URL. Use the lsEntryName=Heavy+Usage+Subscribers parameter to refer to this report. Remember to replace all spaces with the plus “+” sign and to replace the ampersand characters “&” with text entity (&amp;) to avoid XML processing errors.
The report will be saved in a file named Heavy_Usage_Subscribers.pdf on the server's disk in the directory <DELTA_ROOT>\temp\ .

Callout number 3 Transferring the file to an FTP site.
The parameters of this command include an FTP command (PUT), credentials and server address, and local and remote paths. Note the difference in the way local and remote paths are defined.

Callout number 4 Sending the file by email.
To send the file by email, configure an open-relay SMTP server in Settings ► Sending Mail (MAIL_HOST property).
This step may be unnecessary if a rule has been defined on the report server in Report ► Load/Organize. Use this feature when, for example, the email address is meant to be different from the one defined in the report server globally. These parameters set the attachment file path and name, addressee, email subject, and the attachment's MIME type.

XML schema of the task definition file

All task files are XML files that are validated with a schema. NAM Server installations contain an XML schema that defines valid blocks of XML task documents.

The schema file is named tasksXMLSchema.xsd and can be found in the config directory on the report server. The default path is: <installation directory>\config\tasksXMLSchema.xsd.

When using an XML editor that is schema-aware, use this file to help create and validate newly created tasks. Check the validity of any of the tasks that were created in the past or create tasks from scratch and use the editor to help avoid errors.

Example of the XML Schema of a task definition file


<schema xmlns="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.example.org/tasksXMLSchema" xmlns:tns="http://www.example.org/tasksXMLSchema">

  <element name="tasks">
    <complexType>
      <sequence>

        <!-- task definition -->
        <element name="task" maxOccurs="unbounded" minOccurs="0">
          <complexType>

            <sequence>

              <!-- command definition -->
              <element name="command" maxOccurs="unbounded" minOccurs="1">
                <complexType>

                  <sequence>

                    <element name="class" maxOccurs="1" minOccurs="1"><simpleType><restriction base="string">
                      <pattern value="([a-zA-Z0-9_.])*"/>
                    </restriction></simpleType></element>

                    <element name="param" type="string" maxOccurs="unbounded" minOccurs="0"></element>

                  </sequence>

                  <attribute name="ID" use="required"><simpleType><restriction base="string">
                    <pattern value="([a-zA-Z0-9_])*"/>
                  </restriction></simpleType></attribute>

                  <attribute name="name" type="string" use="optional"/>

                  <attribute name="timeout" use="optional"><simpleType><restriction base="string">
                    <pattern value="[0-9][0-9]:[0-9][0-9]"/>
                  </restriction></simpleType></attribute>

                </complexType>
              </element>

            </sequence>

            <!-- task attributes -->
            <attribute name="ID" use="required"><simpleType><restriction base="string">
              <pattern value="([a-zA-Z0-9_])*"/>
            </restriction></simpleType></attribute>

            <attribute name="name" type="string" use="optional"/>

            <attribute name="mergeMode" use="optional"><simpleType><restriction base="string">
              <enumeration value="NONE"/>
              <enumeration value="JOIN"/>
              <enumeration value="REPLACE"/>
              <enumeration value="REMOVE"/>
            </restriction></simpleType></attribute>

            <attribute name="timeLine" use="required"><simpleType><restriction base="string">
              <enumeration value="SERVER"/>
              <enumeration value="SAMPLES"/>
            </restriction></simpleType></attribute>

            <attribute name="periodType" use="required"><simpleType><restriction base="string">
              <enumeration value="MINUTE"/>
              <enumeration value="HOUR"/>
              <enumeration value="DAY"/>
              <enumeration value="WEEK"/>
              <enumeration value="MONTH"/>
            </restriction></simpleType></attribute>

            <attribute name="period" use="required"><simpleType><restriction base="long">
              <minExclusive value="0"/>
            </restriction></simpleType></attribute>

            <attribute name="offsetTime" use="optional"><simpleType><restriction base="string">
              <pattern value="([0-9][0-9])*:[0-9][0-9]:[0-9][0-9]"/>
            </restriction></simpleType></attribute>

            <attribute name="timeout" use="optional"><simpleType><restriction base="string">
              <pattern value="[0-9][0-9]:[0-9][0-9]"/>
            </restriction></simpleType></attribute>

            <attribute name="recoverable" type="boolean" use="optional"/>

            <attribute name="threadFromCmd" use="optional"><simpleType><restriction base="string">
              <pattern value="([a-zA-Z0-9_])*"/>
            </restriction></simpleType></attribute>

          </complexType>
        </element>
      </sequence>
    </complexType>
  </element>
</schema>