Project

General

Profile

ThreatScript Definitions » History » Version 23

« Previous - Version 23/26 (diff) - Next » - Current version
Luke Murphey, 05/28/2011 07:44 PM


ThreatScript Definitions

ThreatScript Definitions are written in ECMAScript (basically the same as JavaScript). The ThreatScript definitions return a Result object which indicates whether a match was observed.

ThreatScript Example

Below is an example of a ThreatScript that triggers if the web-page has a form element.

/*
 * Name: Example.General.Has_Form_Tag
 * Version: 1
 * ID: 1000000
 * Message: Indicates if the page has as a form tag
 * Severity: Low
 * Reference: url,threatfactor.com
 */

importPackage(Packages.ThreatScript);
importPackage(Packages.HTTP);

function analyze( httpResponse, variables, environment ){

    var parser = httpResponse.getDocumentParser();
    var location = new URL( httpResponse.getLocation() );

    //Get a list of all script tags
    var tagNameFilter = new TagNameFilter("form");
    var nodesList = parser.extractAllNodesThatMatch(tagNameFilter); 
        if( nodesList.size() > 0 ){
         return new Result( true, "A form was detected" );
    }

    return new Result( false, "No forms detected" );
}

Analysis Function

ThreatScripts must provide an analyze function that takes 3 arguments:

Name Type Note
httpResponse HttpResponseData See source:trunk/src/net/lukemurphey/nsia/scan/HttpResponseData.java
variables Variables See source:trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Variables.java
environment Environment See source:trunk/src/net/lukemurphey/nsia/scan/ScriptDefinition.java#L605

Saving / Loading Saved Data (using the environment object)

ThreatScripts can save and load data using the environment object. The environment allows the script to recall information between runs of the definition for a given URL and rule.

Retrieving Values

Values can be retrieved using the get function of the environment object. Call getValue on the returned object to get the value. Below is an example:

var saved = environment.get("ValueName");
var value = saved.getValue();

Saving Values

Values can be saved using the set function of the environment object. Optionally, the set argument accepts a boolean indicating if the provided value should only be returned when the definition is running against the same URL. Below is an example:

var test = "1234ABCD";

//Will only be returned when the given definition is executed against the current URL:
environment.set("ValueName", test);

//Will be returned when the given definition is executed against any URL (within the given rule):
environment.set("ValueName", test, false);

Saving / Loading Collections

If your script needs to load collections then you ought to use the Vector class since it will perform the necessary conversions that are required to make sure that the contents are the same when loaded.

v = new Vector();
v.push("one");
v.get(0);
v.pop();
v.remove();
environment.set("Vector", test);

Baseline Function

ThreatScripts may declare a baseline function that will allow the definition to be configured to baseline itself against the previous set of scan results. The baseline function is called by NSIA when a user presses the baseline method for a rule. The objective of the baseline function is to view the provided scan results and ignore the particular finding for the given resource in the future. For example, a definition that triggers when the hash of the web-page changes may define a baseline function that causes it to not trigger unless the web-page hashes changes to yet another value.

Below is an example:

function baseline( environment ){
    var previousValue = environment.get("LastObservedHash");

    if( previousValue != null && previousValue.getValue() != null ){
        environment.set("Hash", previousValue.getValue() );
    }

    return true;
}

Link Extraction

ThreatScript definitions can add URLs to the list that will be scanned. This allows ThreatScripts to parse content and enumerate other content that is linked. URLs can be added by calling addURL on the Result object or by specifying the URLs in the Result constructor. See below for an example:

importPackage(Packages.ThreatScript);
importPackage(Packages.HTTP);

function analyze( httpResponse, operation, environment ){
    a = new Array();
    a[0] = new URL("http://mydomain.com/");
    a[1] = new URL("http://someotherdomain.com/");

    // You can add URLs via the results constructor (or add them manually via the "addURL" methods) 
    result = new Result( false, "Two URLs extracted", a);

    // This URL will not be scanned unless it matches the domain restriction in the rule
    result.addURL( new URL("http://mydomain.com/") );

    // This URL will be scanned regardless of whether it matches the domain restriction in the rule
    result.addURL( new URL("http://someotherdomain.com"), true);
    return result;
}

Terminate Function

ThreatScripts can also declare a terminate function that allows the scan engine to terminate the definition it execution exceeds the maximum execution time.

Below is an example of a (useless) rule that uses the terminate function to stop execution:

importPackage(Packages.ThreatScript);

var keep_going = true;

function analyze( httpResponse, operation, environment ){

    while(keep_going ){
        //Infinite loop
    }

    return new Result( false, "Definition did not match the input");
}

function terminate(){
    keep_going = false; //Will cause the analysis to stop
}

Meta-Data

ThreatScripts must provide a meta-data that indicates the following information:

Name Valid Input Notes
Name <category>.<sub_category>.<definition_name>
Version integer Should be incremented each time the definition is updated
ID integer Must be 1000000 or greater (only official definitions can be less than 1000000)
Message message to be displayed when definition matches
Severity Either: Low, Medium or High
Invasive Either: True or False (this argument is optional)

This meta-data is provided in a comment as name-value pairs (see above ThreatScript example).

Definition Name

Definitions must have a name that is composed of three parts: a category, sub-category, and name. The part are separated by periods (like "category.subcategory.name"). The category and sub-category are important because the user can create exceptions and disable definitions by category or sub-category. Therefore, it is important that you use category names that makes sense; otherwise, the user may disable or enable your definition inadvertently.

In some cases a definition may be created for a specific site-group or rule. It is recommended that the definition be named such it is clear it is site-group or rule specific. Oftentimes, this done by appending an underscore to the category name (e.g. "_Acme.InformationLeak.ConfidentialDocumentFound").

Definition Severity

Definition severities are defined per the below table:

Severity Definition
Low An issue that may represent a problem
Medium Reasonably likely of being an issue
High High likelihood of being a problem and/or are an issue with substantial impact

Definition References

References can be defined to point users to additional information. The references are formatted with the prefix before the argument that is used to create the argument (e.g. "bid,39545" creates a link to BugTraq writeup for issue 39545).

Generic URLs can be created with the url prefix (e.g. "url,google.com" creates a link to Google.com).

The following types of references exist:

Name Prefix Links To
BuqTraq bugtraq http://www.securityfocus.com/bid
CVE cve http://cve.mitre.org/cve/
Nessus nessus http://www.nessus.org/plugins/index.php?view=all
Arachnids arachnids (URL is now defunct, don't use this anymore)
McAfee mcafee http://vil.nai.com/vil/default.aspx
URL url
OSVDB osvdb http://osvdb.org/show/osvdb
USN usn http://www.ubuntu.com/usn/
Milw0rm milworm http://milw0rm.com/exploits
Secunia secunia http://secunia.com/advisories/
RHSA rhsa http://rhn.redhat.com/errata/
Microsoft KB microsoft_kb http://support.microsoft.com/kb/
Microsoft Security Bulletin microsoft_bulletin http://www.microsoft.com/technet/security/bulletin/

Note that definition references are defined a comment block with the "Reference:" as a prefix (Example: "// Reference: url,threatfactor.com").

Available Packages

A series of packages are available to ThreatScripts in order to perform analysis.

Package Class Description
HTTP URL Same as java.net.URL
TagNameFilter See http://htmlparser.sourceforge.net/javadoc/org/htmlparser/filters/TagNameFilter.html
GetRequest Web-client for performing HTTP Get requests
PostRequest Web-client for performing HTTP Post requests
DeleteRequest Web-client for performing HTTP Delete requests
PutRequest Web-client for performing HTTP Put requests
TraceRequest Web-client for performing HTTP Trace requests
HeadRequest Web-client for performing HTTP Head requests
OptionsRequest Web-client for performing HTTP Option requests
<default> StringUtils Provides a trim function for Strings, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/StringUtils.java
Debug Provides method that allows scripts to create log messages, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Debug.java
ThreatScript Result Indicates the results of analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Result.java
DataAnalysis Provides functions useful for analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/ScriptSignatureUtils.java
Vector Provides a storage class for scripts, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Vector.java

Debugging ThreatScripts

ThreatScripts can create event log messages by using the sendMessage() function in the Debug class. Simply call Debug.sendMessage with a string as an argument to create an event log message. The event log messages can be viewed in the event log for NSIA.

Generally, script created log messages are used only for debugging and should be disabled on rules you want to use in production.

General Notes When Writing Definitions

ThreatScript Maximum Runtime

ThreatScript definitions are forceably terminated by the scan engine if the script runs for longer than 10 seconds. Thus, it is important to write definitions that can complete within the timeframe alloted; otherwise, the definition will be flagged as having an error.

Maximum Data Size

The scan engine only provides the first 1 MB of the data observed to the scan engine. Therefore, do not design ThreatScripts that won't work if only the first 1 MB of a larger file is provided.