ThreatFactor NSIA

h1. ThreatScript Definitions

ThreatScript Definitions are written in ECMAScript (basically the same as JavaScript). The ThreatScript definitions return a Result object which indicates whether a match was observed.

h2. ThreatScript Example

Below is an example of a ThreatScript that triggers if the web-page has a form element.

<pre><code class="javascript">

/*

 * Name: Example.General.Has_Form_Tag

 * Version: 1

 * ID: 1000000

 * Message: Indicates if the page has as a form tag

 * Severity: Low

 * Reference: url,threatfactor.com

 */

importPackage(Packages.ThreatScript);

importPackage(Packages.HTTP);

function analyze( httpResponse, variables, environment ){

	var parser = httpResponse.getDocumentParser();

	var location = new URL( httpResponse.getLocation() );

	//Get a list of all script tags

	var tagNameFilter = new TagNameFilter("form");

	var nodesList = parser.extractAllNodesThatMatch(tagNameFilter); 

        if( nodesList.size() > 0 ){

	     return new Result( true, "A form was detected" );

	}

	return new Result( false, "No forms detected" );

}

</code>

</pre>

h2. Analysis Function

ThreatScripts must provide an analyze function that takes 3 arguments:

| *Name*       | *Type*            | *Note*                                                                           |

| httpResponse | HttpResponseData  | See source:trunk/src/net/lukemurphey/nsia/scan/HttpResponseData.java             |

| variables    | Variables         | See source:trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Variables.java  |

| environment  | Environment       | See source:trunk/src/net/lukemurphey/nsia/scan/ScriptDefinition.java#L605        |

h2. Saving / Loading Saved Data (using the environment object)

ThreatScripts can save and load data using the environment object. The environment allows the script to recall information between runs of the definition for a given URL and rule.

h3. Retrieving Values

Values can be retrieved using the _get_ function of the environment object. Call _getValue_ on the returned object to get the value. Below is an example:

<pre><code class="javascript">

var saved = environment.get("ValueName");

var value = saved.getValue();

</code></pre>

h3. Saving Values

Values can be saved using the _set_ function of the environment object. Optionally, the set argument accepts a boolean indicating if the provided value should only be returned when the definition is running against the same URL. Below is an example:

<pre><code class="javascript">

var test = "1234ABCD";

//Will only be returned when the given definition is executed against the current URL:

environment.set("ValueName", test);

//Will be returned when the given definition is executed against the any URL (within the given rule):

environment.set("ValueName", test, false);

</code></pre>

h3. Saving / Loading Collections

If your script needs to load collections then you ought to use the Vector class since it will perform the necessary conversions that are required to make sure that the contents are the same when loaded.

<pre><code class="javascript">

v = new Vector();

v.push("one");

v.get(0);

v.pop();

v.remove();

environment.set("Vector", test);

</code></pre>

h2. Baseline Function

ThreatScripts may declare a baseline function that will allow the definition to be configured to baseline itself against the previous set of scan results. The baseline function is called by NSIA when a user presses the baseline method for a rule. The objective of the baseline function is to view the provided scan results and ignore the particular finding for the given resource in the future. For example, a definition that triggers when the hash of the web-page changes may define a baseline function that causes it to not trigger unless the web-page hashes changes to yet another value.

Below is an example:

<pre>

<code class="javascript">

function baseline( environment ){

	var previousValue = environment.get("LastObservedHash");

	if( previousValue != null && previousValue.getValue() != null ){

		environment.set("Hash", previousValue.getValue() );

	}

	return true;

}

</code>

</pre>

h2. Link Extraction

ThreatScript definitions can add URLs to the list that will be scanned. This allows ThreatScripts to parse content and enumerate other content that is linked. URLs can be added by calling addURL on the Result object or by specifying the URLs in the Result constructor. See below for an example:

<pre>

<code class="javascript">

importPackage(Packages.ThreatScript);

importPackage(Packages.HTTP);

function analyze( httpResponse, operation, environment ){

    a = new Array();

    a[0] = new URL("http://mydomain.com/");

    a[1] = new URL("http://someotherdomain.com/");

    // You can add URLs via the results constructor (or add them manually via the "addURL" methods) 

    result = new Result( false, "Two URLs extracted", a);

    // This URL will not be scanned unless it matches the domain restriction in the rule

    result.addURL( new URL("http://mydomain.com/") );

    // This URL will be scanned regardless of whether it matches the domain restriction in the rule

    result.addURL( new URL("http://someotherdomain.com"), true);

    return result;

}

</code>

</pre>

h2. Terminate Function

ThreatScripts can also declare a terminate function that allows the scan engine to terminate the definition it execution exceeds the maximum execution time.

Below is an example of a (useless) rule that uses the terminate function to stop execution:

<pre>

<code class="javascript">

importPackage(Packages.ThreatScript);

var keep_going = true;

function analyze( httpResponse, operation, environment ){

    while(keep_going ){

        //Infinite loop

    }

    return new Result( false, "Definition did not match the input");

}

function terminate(){

    keep_going = false; //Will cause the analysis to stop

}

</code>

</pre>

h2. Meta-Data

ThreatScripts must provide a meta-data that indicates the following information:

| *Name*  | *Valid Input*                                      | *Notes* |

| Name    | <category>.<sub_category>.<definition_name>        |         |

| Version | integer                                            | Should be incremented each time the definition is updated |

| ID      | integer                                            | Must be 1000000 or greater (only official definitions can be less than 1000000)        |

| Message | message to be displayed when definition matches    |         |

| Severity| Either: Low, Medium or High                        |         |

| Invasive| Either: True or False (this argument is optional)  |         |

This meta-data is provided in a comment as name-value pairs (see above ThreatScript example).

h3. Definition Name

{{include(Definition_Naming_Convention)}}

h3. Definition Severity

{{include(Definition_Severity)}}

h3. Definition References 

{{include(Definition_References)}}

Note that definition references are defined a comment block with the "Reference:" as a prefix (Example: "// Reference: url,threatfactor.com").

h2. Available Packages

A series of packages are available to ThreatScripts in order to perform analysis.

| *Package*            | *Class*            | *Description*                                     |

|/9.HTTP               | URL                | Same as java.net.URL                              |

                       | TagNameFilter      | See http://htmlparser.sourceforge.net/javadoc/org/htmlparser/filters/TagNameFilter.html |

                       | GetRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Get requests  |

                       | PostRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Post requests |

                       | DeleteRequest      | [[ThreatScript Web client|Web-client]] for performing HTTP Delete requests |

                       | PutRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Put requests |

                       | TraceRequest       | [[ThreatScript Web client|Web-client]] for performing HTTP Trace requests |

                       | HeadRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Head requests |

                       | OptionsRequest     | [[ThreatScript Web client|Web-client]] for performing HTTP Option requests |

|/2.<default>          | StringUtils        | Provides a trim function for Strings, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/StringUtils.java              |

| Debug                | Provides method that allows scripts to create log messages, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Debug.java              |

|/3.ThreatScript       | Result             | Indicates the results of analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Result.java                 |

                       | DataAnalysis       | Provides functions useful for analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/ScriptSignatureUtils.java            |

                       | Vector             | Provides a storage class for scripts, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Vector.java            |

h2. Debugging ThreatScripts

ThreatScripts can create event log messages by using the sendMessage() function in the Debug class. Simply call _Debug.sendMessage_ with a string as an argument to create an event log message. The event log messages can be viewed in the event log for NSIA.

Generally, script created log messages are used only for debugging and should be disabled on rules you want to use in production.

h2. General Notes When Writing Definitions

h3. ThreatScript Maximum Runtime

ThreatScript  definitions are forceably terminated by the scan engine if the script runs for longer than 10 seconds. Thus, it is important to write definitions that can complete within the timeframe alloted; otherwise, the definition will be flagged as having an error.

h3. Maximum Data Size

The scan engine only provides the first 1 MB of the data observed to the scan engine. Therefore, do not design ThreatScripts that won't work if only the first 1 MB of a larger file is provided.