Project

General

Profile

ThreatScript Definitions » History » Version 18

Luke Murphey, 09/29/2010 10:30 PM

1 1 Luke Murphey
h1. ThreatScript Definitions
2 1 Luke Murphey
3 1 Luke Murphey
ThreatScript Definitions are written in ECMAScript (basically the same as JavaScript). The ThreatScript definitions return a Result object which indicates whether a match was observed.
4 1 Luke Murphey
5 1 Luke Murphey
h2. ThreatScript Example
6 1 Luke Murphey
7 1 Luke Murphey
Below is an example of a ThreatScript that triggers if the web-page has a form element.
8 1 Luke Murphey
9 1 Luke Murphey
<pre><code class="javascript">
10 1 Luke Murphey
/*
11 1 Luke Murphey
 * Name: Example.General.Has_Form_Tag
12 1 Luke Murphey
 * Version: 1
13 1 Luke Murphey
 * ID: 1000000
14 1 Luke Murphey
 * Message: Indicates if the page has as a form tag
15 1 Luke Murphey
 * Severity: Low
16 11 Luke Murphey
 * Reference: url,threatfactor.com
17 1 Luke Murphey
 */
18 1 Luke Murphey
19 1 Luke Murphey
importPackage(Packages.ThreatScript);
20 1 Luke Murphey
importPackage(Packages.HTTP);
21 1 Luke Murphey
22 8 Luke Murphey
function analyze( httpResponse, variables, environment ){
23 1 Luke Murphey
24 1 Luke Murphey
	var parser = httpResponse.getDocumentParser();
25 1 Luke Murphey
	var location = new URL( httpResponse.getLocation() );
26 1 Luke Murphey
27 1 Luke Murphey
	//Get a list of all script tags
28 1 Luke Murphey
	var tagNameFilter = new TagNameFilter("form");
29 1 Luke Murphey
	var nodesList = parser.extractAllNodesThatMatch(tagNameFilter); 
30 1 Luke Murphey
        if( nodesList.size() > 0 ){
31 1 Luke Murphey
	     return new Result( true, "A form was detected" );
32 1 Luke Murphey
	}
33 1 Luke Murphey
        
34 1 Luke Murphey
	return new Result( false, "No forms detected" );
35 1 Luke Murphey
}
36 1 Luke Murphey
</code>
37 1 Luke Murphey
</pre>
38 2 Luke Murphey
39 3 Luke Murphey
h2. Analysis Function
40 3 Luke Murphey
41 12 Luke Murphey
ThreatScripts must provide an analyze function that takes 3 arguments:
42 3 Luke Murphey
43 7 Luke Murphey
| *Name*       | *Type*            | *Note*                                                                           |
44 7 Luke Murphey
| httpResponse | HttpResponseData  | See source:trunk/src/net/lukemurphey/nsia/scan/HttpResponseData.java             |
45 7 Luke Murphey
| variables    | Variables         | See source:trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Variables.java  |
46 7 Luke Murphey
| environment  | Environment       | See source:trunk/src/net/lukemurphey/nsia/scan/ScriptDefinition.java#L605        |
47 1 Luke Murphey
48 16 Luke Murphey
h2. Saving / Loading Saved Data (using the environment object)
49 16 Luke Murphey
50 16 Luke Murphey
ThreatScripts can save and load data using the environment object. The environment allows the script to recall information between runs of the definition for a given URL and rule.
51 16 Luke Murphey
52 16 Luke Murphey
h3. Retrieving Values
53 16 Luke Murphey
54 16 Luke Murphey
Values can be retrieved using the _get_ function of the environment object. Call _getValue_ on the returned object to get the value. Below is an example:
55 16 Luke Murphey
56 16 Luke Murphey
<pre><code class="javascript">
57 16 Luke Murphey
var saved = environment.get("ValueName");
58 16 Luke Murphey
var value = saved.getValue();
59 16 Luke Murphey
</code></pre>
60 16 Luke Murphey
61 16 Luke Murphey
h3. Saving Values
62 16 Luke Murphey
63 16 Luke Murphey
Values can be saved using the _set_ function of the environment object. Optionally, the set argument accepts a boolean indicating if the provided value should only be returned when the definition is running against the same URL. Below is an example:
64 16 Luke Murphey
65 16 Luke Murphey
<pre><code class="javascript">
66 16 Luke Murphey
var test = "1234ABCD";
67 17 Luke Murphey
68 17 Luke Murphey
//Will only be returned when the given definition is executed against the current URL:
69 17 Luke Murphey
environment.set("ValueName", test);
70 17 Luke Murphey
71 17 Luke Murphey
//Will be returned when the given definition is executed against the any URL (within the given rule):
72 17 Luke Murphey
environment.set("ValueName", test, false);
73 16 Luke Murphey
</code></pre>
74 16 Luke Murphey
75 1 Luke Murphey
h2. Baseline Function
76 1 Luke Murphey
77 4 Luke Murphey
ThreatScripts may declare a baseline function that will allow the definition to be configured to baseline itself against the previous set of scan results. The baseline function is called by NSIA when a user presses the baseline method for a rule. The objective of the baseline function is to view the provided scan results and ignore the particular finding for the given resource in the future. For example, a definition that triggers when the hash of the web-page changes may define a baseline function that causes it to not trigger unless the web-page hashes changes to yet another value.
78 4 Luke Murphey
79 4 Luke Murphey
Below is an example:
80 4 Luke Murphey
81 4 Luke Murphey
<pre>
82 4 Luke Murphey
<code class="javascript">
83 4 Luke Murphey
function baseline( environment ){
84 4 Luke Murphey
	var previousValue = environment.get("LastObservedHash");
85 4 Luke Murphey
86 4 Luke Murphey
	if( previousValue != null && previousValue.getValue() != null ){
87 4 Luke Murphey
		environment.set("Hash", previousValue.getValue() );
88 4 Luke Murphey
	}
89 4 Luke Murphey
90 4 Luke Murphey
	return true;
91 4 Luke Murphey
}
92 4 Luke Murphey
</code>
93 4 Luke Murphey
</pre>
94 4 Luke Murphey
95 14 Luke Murphey
h2. Terminate Function
96 3 Luke Murphey
97 14 Luke Murphey
ThreatScripts can also declare a terminate function that allows the scan engine to terminate the definition it execution exceeds the maximum execution time.
98 14 Luke Murphey
99 14 Luke Murphey
Below is an example of a (useless) rule that uses the terminate function to stop execution:
100 14 Luke Murphey
101 14 Luke Murphey
<pre>
102 14 Luke Murphey
<code class="javascript">
103 14 Luke Murphey
importPackage(Packages.ThreatScript);
104 14 Luke Murphey
105 14 Luke Murphey
var keep_going = true;
106 14 Luke Murphey
107 14 Luke Murphey
function analyze( httpResponse, operation, environment ){
108 14 Luke Murphey
109 14 Luke Murphey
    while(keep_going ){
110 14 Luke Murphey
        //Infinite loop
111 14 Luke Murphey
    }
112 14 Luke Murphey
113 14 Luke Murphey
    return new Result( false, "Definition did not match the input");
114 14 Luke Murphey
}
115 14 Luke Murphey
116 14 Luke Murphey
function terminate(){
117 14 Luke Murphey
    keep_going = false; //Will cause the analysis to stop
118 14 Luke Murphey
}
119 14 Luke Murphey
</code>
120 14 Luke Murphey
</pre>
121 3 Luke Murphey
122 1 Luke Murphey
h2. Meta-Data
123 1 Luke Murphey
124 1 Luke Murphey
ThreatScripts must provide a meta-data that indicates the following information:
125 1 Luke Murphey
126 3 Luke Murphey
| *Name*  | *Valid Input*                                      | *Notes* |
127 3 Luke Murphey
| Name    | <category>.<sub_category>.<definition_name>        |         |
128 3 Luke Murphey
| Version | integer                                            | Should be incremented each time the definition is updated |
129 3 Luke Murphey
| ID      | integer                                            | Must be 1000000 or greater (only official definitions can be less than 1000000)        |
130 3 Luke Murphey
| Message | message to be displayed when definition matches    |         |
131 3 Luke Murphey
| Severity| Either: Low, Medium or High                        |         |
132 3 Luke Murphey
| Invasive| Either: True or False (this argument is optional)  |         |
133 1 Luke Murphey
134 1 Luke Murphey
This meta-data is provided in a comment as name-value pairs (see above ThreatScript example).
135 1 Luke Murphey
136 9 Luke Murphey
h3. Definition Name
137 8 Luke Murphey
138 1 Luke Murphey
{{include(Definition_Naming_Convention)}}
139 1 Luke Murphey
140 9 Luke Murphey
h3. Definition Severity
141 9 Luke Murphey
142 9 Luke Murphey
{{include(Definition_Severity)}}
143 11 Luke Murphey
144 11 Luke Murphey
h3. Definition References 
145 11 Luke Murphey
146 11 Luke Murphey
{{include(Definition_References)}}
147 11 Luke Murphey
148 11 Luke Murphey
Note that definition references are defined a comment block with the "Reference:" as a prefix (Example: "// Reference: url,threatfactor.com").
149 9 Luke Murphey
150 3 Luke Murphey
h2. Available Packages
151 1 Luke Murphey
152 3 Luke Murphey
A series of packages are available to ThreatScripts in order to perform analysis.
153 3 Luke Murphey
154 1 Luke Murphey
| *Package*            | *Class*            | *Description*                                     |
155 13 Luke Murphey
|/9.HTTP               | URL                | Same as java.net.URL                              |
156 1 Luke Murphey
                       | TagNameFilter      | See http://htmlparser.sourceforge.net/javadoc/org/htmlparser/filters/TagNameFilter.html |
157 15 Luke Murphey
                       | GetRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Get requests  |
158 15 Luke Murphey
                       | PostRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Post requests |
159 15 Luke Murphey
                       | DeleteRequest      | [[ThreatScript Web client|Web-client]] for performing HTTP Delete requests |
160 15 Luke Murphey
                       | PutRequest         | [[ThreatScript Web client|Web-client]] for performing HTTP Put requests |
161 15 Luke Murphey
                       | TraceRequest       | [[ThreatScript Web client|Web-client]] for performing HTTP Trace requests |
162 15 Luke Murphey
                       | HeadRequest        | [[ThreatScript Web client|Web-client]] for performing HTTP Head requests |
163 15 Luke Murphey
                       | OptionsRequest     | [[ThreatScript Web client|Web-client]] for performing HTTP Option requests |
164 13 Luke Murphey
|/2.<default>          | StringUtils        | Provides a trim function for Strings, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/StringUtils.java              |
165 13 Luke Murphey
| Debug                | Provides method that allows scripts to create log messages, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Debug.java              |
166 18 Luke Murphey
|/2.ThreatScript       | Result             | Indicates the results of analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Result.java                 |
167 18 Luke Murphey
                       | DataAnalysis       | Provides functions useful for analysis, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/ScriptSignatureUtils.java            |
168 18 Luke Murphey
                       | Vector             | Provides a storage class for scripts, see source:/trunk/src/net/lukemurphey/nsia/scan/scriptenvironment/Vector.java            |
169 1 Luke Murphey
170 1 Luke Murphey
171 1 Luke Murphey
h2. Debugging ThreatScripts
172 8 Luke Murphey
173 8 Luke Murphey
ThreatScripts can create event log messages by using the sendMessage() function in the Debug class. Simply call _Debug.sendMessage_ with a string as an argument to create an event log message. The event log messages can be viewed in the event log for NSIA.
174 8 Luke Murphey
175 8 Luke Murphey
Generally, script created log messages are used only for debugging and should be disabled on rules you want to use in production.
176 8 Luke Murphey
177 8 Luke Murphey
h2. General Notes When Writing Definitions
178 8 Luke Murphey
179 8 Luke Murphey
h3. ThreatScript Maximum Runtime
180 8 Luke Murphey
181 8 Luke Murphey
ThreatScript  definitions are forceably terminated by the scan engine if the script runs for longer than 10 seconds. Thus, it is important to write definitions that can complete within the timeframe alloted; otherwise, the definition will be flagged as having an error.
182 8 Luke Murphey
183 8 Luke Murphey
h3. Maximum Data Size
184 8 Luke Murphey
185 8 Luke Murphey
The scan engine only provides the first 1 MB of the data observed to the scan engine. Therefore, do not design ThreatScripts that won't work if only the first 1 MB of a larger file is provided.