Project

General

Profile

ThreatScript Definitions » History » Version 16

Luke Murphey, 09/24/2010 08:30 PM

1 1 Luke Murphey
h1. ThreatScript Definitions
2 1 Luke Murphey
3 1 Luke Murphey
ThreatScript Definitions are written in ECMAScript (basically the same as JavaScript). The ThreatScript definitions return a Result object which indicates whether a match was observed.
4 1 Luke Murphey
5 1 Luke Murphey
h2. ThreatScript Example
6 1 Luke Murphey
7 1 Luke Murphey
Below is an example of a ThreatScript that triggers if the web-page has a form element.
8 1 Luke Murphey
9 1 Luke Murphey
<pre><code class="javascript">
10 1 Luke Murphey
/*
11 1 Luke Murphey
 * Name: Example.General.Has_Form_Tag
12 1 Luke Murphey
 * Version: 1
13 1 Luke Murphey
 * ID: 1000000
14 1 Luke Murphey
 * Message: Indicates if the page has as a form tag
15 1 Luke Murphey
 * Severity: Low
16 11 Luke Murphey
 * Reference: url,threatfactor.com
17 1 Luke Murphey
 */
18 1 Luke Murphey
19 1 Luke Murphey
importPackage(Packages.ThreatScript);
20 1 Luke Murphey
importPackage(Packages.HTTP);
21 1 Luke Murphey
22 8 Luke Murphey
function analyze( httpResponse, variables, environment ){
23 1 Luke Murphey
24 1 Luke Murphey
	var parser = httpResponse.getDocumentParser();
25 1 Luke Murphey
	var location = new URL( httpResponse.getLocation() );
26 1 Luke Murphey
27 1 Luke Murphey
	//Get a list of all script tags
28 1 Luke Murphey
	var tagNameFilter = new TagNameFilter("form");
29 1 Luke Murphey
	var nodesList = parser.extractAllNodesThatMatch(tagNameFilter); 
30 1 Luke Murphey
        if( nodesList.size() > 0 ){
31 1 Luke Murphey
	     return new Result( true, "A form was detected" );
32 1 Luke Murphey
	}
33 1 Luke Murphey
        
34 1 Luke Murphey
	return new Result( false, "No forms detected" );
35 1 Luke Murphey
}
36 1 Luke Murphey
</code>
37 1 Luke Murphey
</pre>
38 2 Luke Murphey
39 3 Luke Murphey
h2. Analysis Function
40 3 Luke Murphey
41 12 Luke Murphey
ThreatScripts must provide an analyze function that takes 3 arguments:
42 3 Luke Murphey
43 7 Luke Murphey
| *Name*       | *Type*            | *Note*                                                                           |
44 7 Luke Murphey
| httpResponse | HttpResponseData  | See source:trunk/src/net/lukemurphey/nsia/scan/HttpResponseData.java             |
45 7 Luke Murphey
| variables    | Variables         | See source:trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Variables.java  |
46 7 Luke Murphey
| environment  | Environment       | See source:trunk/src/net/lukemurphey/nsia/scan/ScriptDefinition.java#L605        |
47 1 Luke Murphey
48 16 Luke Murphey
h2. Saving / Loading Saved Data (using the environment object)
49 16 Luke Murphey
50 16 Luke Murphey
ThreatScripts can save and load data using the environment object. The environment allows the script to recall information between runs of the definition for a given URL and rule.
51 16 Luke Murphey
52 16 Luke Murphey
h3. Retrieving Values
53 16 Luke Murphey
54 16 Luke Murphey
Values can be retrieved using the _get_ function of the environment object. Call _getValue_ on the returned object to get the value. Below is an example:
55 16 Luke Murphey
56 16 Luke Murphey
<pre><code class="javascript">
57 16 Luke Murphey
var saved = environment.get("ValueName");
58 16 Luke Murphey
var value = saved.getValue();
59 16 Luke Murphey
</code></pre>
60 16 Luke Murphey
61 16 Luke Murphey
h3. Saving Values
62 16 Luke Murphey
63 16 Luke Murphey
Values can be saved using the _set_ function of the environment object. Optionally, the set argument accepts a boolean indicating if the provided value should only be returned when the definition is running against the same URL. Below is an example:
64 16 Luke Murphey
65 16 Luke Murphey
<pre><code class="javascript">
66 16 Luke Murphey
var test = "1234ABCD";
67 16 Luke Murphey
environment.set("ValueName", test); //Will only be returned when the given definition is executed against the current URL
68 16 Luke Murphey
environment.set("ValueName", test, false); //Will be returned when the given definition is executed against the any URL (within the given rule)
69 16 Luke Murphey
</code></pre>
70 16 Luke Murphey
71 1 Luke Murphey
h2. Baseline Function
72 1 Luke Murphey
73 4 Luke Murphey
ThreatScripts may declare a baseline function that will allow the definition to be configured to baseline itself against the previous set of scan results. The baseline function is called by NSIA when a user presses the baseline method for a rule. The objective of the baseline function is to view the provided scan results and ignore the particular finding for the given resource in the future. For example, a definition that triggers when the hash of the web-page changes may define a baseline function that causes it to not trigger unless the web-page hashes changes to yet another value.
74 4 Luke Murphey
75 4 Luke Murphey
Below is an example:
76 4 Luke Murphey
77 4 Luke Murphey
<pre>
78 4 Luke Murphey
<code class="javascript">
79 4 Luke Murphey
function baseline( environment ){
80 4 Luke Murphey
	var previousValue = environment.get("LastObservedHash");
81 4 Luke Murphey
82 4 Luke Murphey
	if( previousValue != null && previousValue.getValue() != null ){
83 4 Luke Murphey
		environment.set("Hash", previousValue.getValue() );
84 4 Luke Murphey
	}
85 4 Luke Murphey
86 4 Luke Murphey
	return true;
87 4 Luke Murphey
}
88 4 Luke Murphey
</code>
89 4 Luke Murphey
</pre>
90 4 Luke Murphey
91 14 Luke Murphey
h2. Terminate Function
92 3 Luke Murphey
93 14 Luke Murphey
ThreatScripts can also declare a terminate function that allows the scan engine to terminate the definition it execution exceeds the maximum execution time.
94 14 Luke Murphey
95 14 Luke Murphey
Below is an example of a (useless) rule that uses the terminate function to stop execution:
96 14 Luke Murphey
97 14 Luke Murphey
<pre>
98 14 Luke Murphey
<code class="javascript">
99 14 Luke Murphey
importPackage(Packages.ThreatScript);
100 14 Luke Murphey
101 14 Luke Murphey
var keep_going = true;
102 14 Luke Murphey
103 14 Luke Murphey
function analyze( httpResponse, operation, environment ){
104 14 Luke Murphey
105 14 Luke Murphey
    while(keep_going ){
106 14 Luke Murphey
        //Infinite loop
107 14 Luke Murphey
    }
108 14 Luke Murphey
109 14 Luke Murphey
    return new Result( false, "Definition did not match the input");
110 14 Luke Murphey
}
111 14 Luke Murphey
112 14 Luke Murphey
function terminate(){
113 14 Luke Murphey
    keep_going = false; //Will cause the analysis to stop
114 14 Luke Murphey
}
115 14 Luke Murphey
</code>
116 14 Luke Murphey
</pre>
117 3 Luke Murphey
118 1 Luke Murphey
h2. Meta-Data
119 1 Luke Murphey
120 1 Luke Murphey
ThreatScripts must provide a meta-data that indicates the following information:
121 1 Luke Murphey
122 3 Luke Murphey
| *Name*  | *Valid Input*                                      | *Notes* |
123 3 Luke Murphey
| Name    | <category>.<sub_category>.<definition_name>        |         |
124 3 Luke Murphey
| Version | integer                                            | Should be incremented each time the definition is updated |
125 3 Luke Murphey
| ID      | integer                                            | Must be 1000000 or greater (only official definitions can be less than 1000000)        |
126 3 Luke Murphey
| Message | message to be displayed when definition matches    |         |
127 3 Luke Murphey
| Severity| Either: Low, Medium or High                        |         |
128 3 Luke Murphey
| Invasive| Either: True or False (this argument is optional)  |         |
129 1 Luke Murphey
130 1 Luke Murphey
This meta-data is provided in a comment as name-value pairs (see above ThreatScript example).
131 1 Luke Murphey
132 9 Luke Murphey
h3. Definition Name
133 8 Luke Murphey
134 1 Luke Murphey
{{include(Definition_Naming_Convention)}}
135 1 Luke Murphey
136 9 Luke Murphey
h3. Definition Severity
137 9 Luke Murphey
138 9 Luke Murphey
{{include(Definition_Severity)}}
139 11 Luke Murphey
140 11 Luke Murphey
h3. Definition References 
141 11 Luke Murphey
142 11 Luke Murphey
{{include(Definition_References)}}
143 11 Luke Murphey
144 11 Luke Murphey
Note that definition references are defined a comment block with the "Reference:" as a prefix (Example: "// Reference: url,threatfactor.com").
145 9 Luke Murphey
146 3 Luke Murphey
h2. Available Packages
147 1 Luke Murphey
148 3 Luke Murphey
A series of packages are available to ThreatScripts in order to perform analysis.
149 3 Luke Murphey
150 1 Luke Murphey
| *Package*            | *Class*            | *Description*                                     |
151 13 Luke Murphey
|/9.HTTP               | URL                | Same as java.net.URL                              |
152 1 Luke Murphey
                       | TagNameFilter      | See http://htmlparser.sourceforge.net/javadoc/org/htmlparser/filters/TagNameFilter.html |
153 15 Luke Murphey
                       | GetRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Get requests  |
154 15 Luke Murphey
                       | PostRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Post requests |
155 15 Luke Murphey
                       | DeleteRequest      | [[ThreatScript Web client|Web-client]] for performing HTTP Delete requests |
156 15 Luke Murphey
                       | PutRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Put requests |
157 15 Luke Murphey
                       | TraceRequest       | [[ThreatScript Web client|Web-client]] for performing HTTP Trace requests |
158 15 Luke Murphey
                       | HeadRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Head requests |
159 15 Luke Murphey
                       | OptionsRequest     | [[ThreatScript Web client|Web-client]] for performing HTTP Option requests |
160 13 Luke Murphey
|/2.<default>          | StringUtils        | Provides a trim function for Strings, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/StringUtils.java              |
161 13 Luke Murphey
| Debug                | Provides method that allows scripts to create log messages, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Debug.java              |
162 7 Luke Murphey
|/2.ThreatScript       | Result             | Indicates the results of analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment.Result.java                 |
163 7 Luke Murphey
                       | DataAnalysis       | Provides functions useful for analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/ScriptSignatureUtils.java            |
164 1 Luke Murphey
165 1 Luke Murphey
166 1 Luke Murphey
h2. Debugging ThreatScripts
167 8 Luke Murphey
168 8 Luke Murphey
ThreatScripts can create event log messages by using the sendMessage() function in the Debug class. Simply call _Debug.sendMessage_ with a string as an argument to create an event log message. The event log messages can be viewed in the event log for NSIA.
169 8 Luke Murphey
170 8 Luke Murphey
Generally, script created log messages are used only for debugging and should be disabled on rules you want to use in production.
171 8 Luke Murphey
172 8 Luke Murphey
h2. General Notes When Writing Definitions
173 8 Luke Murphey
174 8 Luke Murphey
h3. ThreatScript Maximum Runtime
175 8 Luke Murphey
176 8 Luke Murphey
ThreatScript  definitions are forceably terminated by the scan engine if the script runs for longer than 10 seconds. Thus, it is important to write definitions that can complete within the timeframe alloted; otherwise, the definition will be flagged as having an error.
177 8 Luke Murphey
178 8 Luke Murphey
h3. Maximum Data Size
179 8 Luke Murphey
180 8 Luke Murphey
The scan engine only provides the first 1 MB of the data observed to the scan engine. Therefore, do not design ThreatScripts that won't work if only the first 1 MB of a larger file is provided.