• ThreatScript Definitions
  • ThreatScript Example
  • Analysis Function
  • Saving / Loading Saved Data (using the environment object)
    • Retrieving Values
    • Saving Values
    • Removing Values
    • Saving / Loading Collections
  • Baseline Function
  • Link Extraction
  • Terminate Function
  • Meta-Data
    • Definition Name
    • Definition Severity
    • Definition References
  • Available Packages
  • Debugging ThreatScripts
  • General Notes When Writing Definitions
    • ThreatScript Maximum Runtime
    • Maximum Data Size

ThreatScript Definitions¶

ThreatScript Definitions are written in ECMAScript (basically the same as JavaScript). The ThreatScript definitions return a Result object which indicates whether a match was observed.

ThreatScript Example¶

Below is an example of a ThreatScript that triggers if the web-page has a form element.

/*
 * Name: Example.General.Has_Form_Tag
 * Version: 1
 * ID: 1000000
 * Message: Indicates if the page has as a form tag
 * Severity: Low
 * Reference: url,threatfactor.com
 */

importPackage(Packages.ThreatScript);
importPackage(Packages.HTTP);

function analyze( httpResponse, variables, environment ){

    var parser = httpResponse.getDocumentParser();
    var location = new URL( httpResponse.getLocation() );

    //Get a list of all script tags
    var tagNameFilter = new TagNameFilter("form");
    var nodesList = parser.extractAllNodesThatMatch(tagNameFilter); 
        if( nodesList.size() > 0 ){
         return new Result( true, "A form was detected" );
    }

    return new Result( false, "No forms detected" );
}

Analysis Function¶

ThreatScripts must provide an analyze function that takes 3 arguments:

Name	Type	Note
httpResponse	HttpResponseData	See source:trunk/src/net/lukemurphey/nsia/scan/HttpResponseData.java
variables	Variables	See source:trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Variables.java
environment	Environment	See source:trunk/src/net/lukemurphey/nsia/scan/ScriptDefinition.java#L605

Note that a couple of static variables are available as well:

Name	Type	Note
scanStarted	Date	Defines when the rule began to scan
scriptStarted	Date	Defines when the ThreatScript began executing

Saving / Loading Saved Data (using the environment object)¶

ThreatScripts can save and load data using the environment object. The environment allows the script to recall information between runs of the definition for a given URL and rule.

Retrieving Values¶

Values can be retrieved using the get function of the environment object. Call getValue on the returned object to get the value. Below is an example:

var saved = environment.get("ValueName");
var value = saved.getValue();

Saving Values¶

Values can be saved using the set function of the environment object. Optionally, the set argument accepts a boolean indicating if the provided value should only be returned when the definition is running against the same URL. Below is an example:

var test = "1234ABCD";

//Will only be returned when the given definition is executed against the current URL:
environment.set("ValueName", test);

//Will be returned when the given definition is executed against any URL (within the given rule):
environment.set("ValueName", test, false);

Removing Values¶

Values can be saved using the remove function of the environment object. Below is an example:

//Removes the object named "RemoveMe" if it exists
environment.remove("RemoveMe");

Saving / Loading Collections¶

If your script needs to load collections then you ought to use the Vector class since it will perform the necessary conversions that are required to make sure that the contents are the same when loaded.

v = new Vector();
v.push("one");
v.get(0);
v.pop();
v.remove();
environment.set("Vector", test);

Baseline Function¶

ThreatScripts may declare a baseline function that will allow the definition to be configured to baseline itself against the previous set of scan results. The baseline function is called by NSIA when a user presses the baseline method for a rule. The objective of the baseline function is to view the provided scan results and ignore the particular finding for the given resource in the future. For example, a definition that triggers when the hash of the web-page changes may define a baseline function that causes it to not trigger unless the web-page hashes changes to yet another value.

Below is an example:

function baseline( environment ){
    var previousValue = environment.get("LastObservedHash");

    if( previousValue != null && previousValue.getValue() != null ){
        environment.set("Hash", previousValue.getValue() );
    }

    return true;
}

Link Extraction¶

ThreatScript definitions can add URLs to the list that will be scanned. This allows ThreatScripts to parse content and enumerate other content that is linked. URLs can be added by calling addURL on the Result object or by specifying the URLs in the Result constructor. See below for an example:

importPackage(Packages.ThreatScript);
importPackage(Packages.HTTP);

function analyze( httpResponse, operation, environment ){
    a = new Array();
    a[0] = new URL("http://mydomain.com/");
    a[1] = new URL("http://someotherdomain.com/");

    // You can add URLs via the results constructor (or add them manually via the "addURL" methods) 
    result = new Result( false, "Two URLs extracted", a);

    // This URL will not be scanned unless it matches the domain restriction in the rule
    result.addURL( new URL("http://mydomain.com/") );

    // This URL will be scanned regardless of whether it matches the domain restriction in the rule
    result.addURL( new URL("http://someotherdomain.com"), true);
    return result;
}

Terminate Function¶

ThreatScripts can also declare a terminate function that allows the scan engine to terminate the definition it execution exceeds the maximum execution time.

Below is an example of a (useless) rule that uses the terminate function to stop execution:

importPackage(Packages.ThreatScript);

var keep_going = true;

function analyze( httpResponse, operation, environment ){

    while(keep_going ){
        //Infinite loop
    }

    return new Result( false, "Definition did not match the input");
}

function terminate(){
    keep_going = false; //Will cause the analysis to stop
}

Meta-Data¶

ThreatScripts must provide a meta-data that indicates the following information:

Name	Valid Input	Notes
Name	<category>.<sub_category>.<definition_name>
Version	integer	Should be incremented each time the definition is updated
ID	integer	Must be 1000000 or greater (only official definitions can be less than 1000000)
Message	message to be displayed when definition matches
Severity	Either: Low, Medium or High
Invasive	Either: True or False (this argument is optional)

This meta-data is provided in a comment as name-value pairs (see above ThreatScript example).

Definition Name¶

Definitions must have a name that is composed of three parts: a category, sub-category, and name. The part are separated by periods (like "category.subcategory.name"). The category and sub-category are important because the user can create exceptions and disable definitions by category or sub-category. Therefore, it is important that you use category names that makes sense; otherwise, the user may disable or enable your definition inadvertently.

In some cases a definition may be created for a specific site-group or rule. It is recommended that the definition be named such it is clear it is site-group or rule specific. Oftentimes, this done by appending an underscore to the category name (e.g. "_Acme.InformationLeak.ConfidentialDocumentFound").

Definition Severity¶

Definition severities are defined per the below table:

Severity	Definition
Low	An issue that may represent a problem
Medium	Reasonably likely of being an issue
High	High likelihood of being a problem and/or are an issue with substantial impact

Definition References¶

References can be defined to point users to additional information. The references are formatted with the prefix before the argument that is used to create the argument (e.g. "bid,39545" creates a link to BugTraq writeup for issue 39545).

Generic URLs can be created with the url prefix (e.g. "url,google.com" creates a link to Google.com).

The following types of references exist:

Name	Prefix	Links To
BuqTraq	bugtraq	http://www.securityfocus.com/bid
CVE	cve	http://cve.mitre.org/cve/
Nessus	nessus	http://www.nessus.org/plugins/index.php?view=all
Arachnids	arachnids	(URL is now defunct, don't use this anymore)
McAfee	mcafee	http://vil.nai.com/vil/default.aspx
URL	url
OSVDB	osvdb	http://osvdb.org/show/osvdb
USN	usn	http://www.ubuntu.com/usn/
Milw0rm	milworm	http://milw0rm.com/exploits
Secunia	secunia	http://secunia.com/advisories/
RHSA	rhsa	http://rhn.redhat.com/errata/
Microsoft KB	microsoft_kb	http://support.microsoft.com/kb/
Microsoft Security Bulletin	microsoft_bulletin	http://www.microsoft.com/technet/security/bulletin/

Note that definition references are defined a comment block with the "Reference:" as a prefix (Example: "// Reference: url,threatfactor.com").

Available Packages¶

A series of packages are available to ThreatScripts in order to perform analysis.

Package	Class	Description
HTTP	URL	Same as java.net.URL
	TagNameFilter	See http://htmlparser.sourceforge.net/javadoc/org/htmlparser/filters/TagNameFilter.html
	GetRequest	Web-client for performing HTTP Get requests
	PostRequest	Web-client for performing HTTP Post requests
	DeleteRequest	Web-client for performing HTTP Delete requests
	PutRequest	Web-client for performing HTTP Put requests
	TraceRequest	Web-client for performing HTTP Trace requests
	HeadRequest	Web-client for performing HTTP Head requests
	OptionsRequest	Web-client for performing HTTP Option requests
<default>	StringUtils	Provides a trim function for Strings, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/StringUtils.java
	Debug	Provides method that allows scripts to create log messages, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Debug.java
ThreatScript	Result	Indicates the results of analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Result.java
	DataAnalysis	Provides functions useful for analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/ScriptSignatureUtils.java
	Vector	Provides a storage class for scripts, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Vector.java

Debugging ThreatScripts¶

ThreatScripts can create event log messages by using the sendMessage() function in the Debug class. Simply call Debug.sendMessage with a string as an argument to create an event log message. The event log messages can be viewed in the event log for NSIA.

Generally, script created log messages are used only for debugging and should be disabled on rules you want to use in production.

General Notes When Writing Definitions¶

ThreatScript Maximum Runtime¶

ThreatScript definitions are forceably terminated by the scan engine if the script runs for longer than 10 seconds. Thus, it is important to write definitions that can complete within the timeframe alloted; otherwise, the definition will be flagged as having an error.

Maximum Data Size¶

The scan engine only provides the first 1 MB of the data observed to the scan engine. Therefore, do not design ThreatScripts that won't work if only the first 1 MB of a larger file is provided.