Project

General

Profile

Setup And Install » History » Version 20

Luke Murphey, 11/08/2018 06:14 AM

1 15 Luke Murphey
{{>toc}}
2 15 Luke Murphey
3 1 Luke Murphey
h1. Setup And Install
4 1 Luke Murphey
5 2 Luke Murphey
Below are instructions for how to run your own instance of TextCritical.
6 2 Luke Murphey
7 1 Luke Murphey
h2. Install Prerequisites
8 1 Luke Murphey
9 2 Luke Murphey
To run TextCritical, you will need:
10 2 Luke Murphey
11 13 Luke Murphey
* "Django 1.8+":https://www.djangoproject.com/download/
12 1 Luke Murphey
* "Python 2.7":http://www.python.org/getit/
13 13 Luke Murphey
* "Grappelli":http://django-grappelli.readthedocs.org/en/latest/quickstart.html#installation
14 1 Luke Murphey
* A database (you can use SQLite to avoid extensive database setup)
15 16 Luke Murphey
* KindleGen (to create MOBI ebook files)
16 19 Luke Murphey
* "lxml":http://lxml.de/installation.html
17 19 Luke Murphey
* "Genshi":http://genshi.edgewall.org/wiki/Download
18 1 Luke Murphey
19 20 Luke Murphey
You might want to setup a virtual environment to run the app in:
20 20 Luke Murphey
21 20 Luke Murphey
<pre>
22 20 Luke Murphey
cd ~/venv/
23 20 Luke Murphey
virtualenv textcritical
24 20 Luke Murphey
</pre>
25 20 Luke Murphey
26 19 Luke Murphey
You can install django, lxml, Genshi and Grappelli by running the build script target "install_dependencies":
27 1 Luke Murphey
28 19 Luke Murphey
<pre>
29 19 Luke Murphey
ant install_dependencies
30 19 Luke Murphey
</pre>
31 19 Luke Murphey
32 16 Luke Murphey
To install KindleGen, install the kindlgegen binary in your path such as /usr/local/bin. You can get it using wget:
33 16 Luke Murphey
34 16 Luke Murphey
<pre>
35 16 Luke Murphey
wget http://kindlegen.s3.amazonaws.com/kindlegen_linux_2.6_i386_v2_9.tar.gz
36 2 Luke Murphey
</pre>
37 10 Luke Murphey
38 1 Luke Murphey
h2. Install TextCritical App Code
39 1 Luke Murphey
40 19 Luke Murphey
Check out the source code from https://github.com/LukeMurphey/textcritical_net
41 2 Luke Murphey
42 2 Luke Murphey
Copy start_here_settings.py to settings.py (under the src directory). Update the settings.py file per your host.
43 2 Luke Murphey
44 10 Luke Murphey
h2. Configure and Initialize Database
45 1 Luke Murphey
46 2 Luke Murphey
Specify a database in settings.py unless you intend to use SQLite.
47 2 Luke Murphey
48 17 Luke Murphey
After configuring the database, initialize the database (from the src directory):
49 2 Luke Murphey
50 1 Luke Murphey
<pre>
51 18 Luke Murphey
python .\manage.py makemigrations
52 17 Luke Murphey
python .\manage.py migrate
53 1 Luke Murphey
</pre>
54 1 Luke Murphey
55 1 Luke Murphey
h2. Import Works
56 1 Luke Murphey
57 1 Luke Murphey
You will need to import the works into the library in order to have something for the site to provide access to. The easy way to do this is to use a prebuilt library. The more difficult way is to import the works yourself.
58 1 Luke Murphey
59 1 Luke Murphey
h3. Using Existing Library
60 1 Luke Murphey
61 1 Luke Murphey
To use an existing library, copy the library.sqlite file to the server. By default, it should be placed in the src directory.
62 1 Luke Murphey
63 1 Luke Murphey
h3. Importing Works
64 1 Luke Murphey
65 1 Luke Murphey
If you want to import works manually, then following the steps below.
66 1 Luke Murphey
67 1 Luke Murphey
First, initialize the library database.
68 1 Luke Murphey
69 1 Luke Murphey
<pre>
70 1 Luke Murphey
python manage.py syncdb --database=library
71 1 Luke Murphey
</pre>
72 1 Luke Murphey
73 2 Luke Murphey
Next, download the relevant works. The Perseus works can be obtained from "Perseus.tufts.edu":http://www.perseus.tufts.edu/hopper/opensource/download. Make sure to get the classics library. Decompress the archive.
74 1 Luke Murphey
75 2 Luke Murphey
Start the import process by running the following command, substituting "/Users/Luke/Perseus_Directory" with the location where you placed the files:
76 1 Luke Murphey
77 1 Luke Murphey
<pre>
78 1 Luke Murphey
python manage.py batch_import_perseus -d "/Users/Luke/Perseus_Directory"
79 1 Luke Murphey
</pre>
80 1 Luke Murphey
81 6 Luke Murphey
The import process may take a while (it takes about 70 minutes on a Core i7 with 8 GB of RAM). *Make sure to set DEBUG to False* before running the import. Otherwise, memory consumption will grow until the import completes.
82 2 Luke Murphey
83 8 Luke Murphey
h3. Indexing Works
84 8 Luke Murphey
85 8 Luke Murphey
You'll need to index the works for the search engine to work properly.
86 8 Luke Murphey
87 8 Luke Murphey
After importing the works, run the following:
88 8 Luke Murphey
89 8 Luke Murphey
<pre>
90 8 Luke Murphey
python manage.py make_search_indexes -c
91 8 Luke Murphey
</pre>
92 8 Luke Murphey
93 1 Luke Murphey
h2. Start Web Server
94 2 Luke Murphey
95 4 Luke Murphey
For production installs, you ought to use a production web-server like Apache. For development, you can use the built in Django web-server:
96 1 Luke Murphey
97 1 Luke Murphey
<pre>
98 1 Luke Murphey
python manage.py runserver 0.0.0.0:8080
99 4 Luke Murphey
</pre>
100 4 Luke Murphey
101 4 Luke Murphey
Alternatively, you can use the included CherryPy server which should be good enough for production use. To use it, start "run_server.py" after setting WEB_SERVER_ADDRESS and WEB_SERVER_PORT in settings.py; example below:
102 4 Luke Murphey
103 4 Luke Murphey
<pre>
104 4 Luke Murphey
<code class="python">
105 4 Luke Murphey
# The address and port to use when using the built-in web-server
106 4 Luke Murphey
WEB_SERVER_ADDRESS = '0.0.0.0' # Use '127.0.0.1' to serve content to localhost only
107 4 Luke Murphey
WEB_SERVER_PORT    = 8080
108 4 Luke Murphey
</code>
109 2 Luke Murphey
</pre>
110 3 Luke Murphey
111 11 Luke Murphey
You can use lighttpd to serve as a reverse proxy to CherryPy. Below is a lighttpd config that will proxy to CherryPy or the Django web-server:
112 11 Luke Murphey
113 11 Luke Murphey
<pre>
114 11 Luke Murphey
# Define the location of text critical here:
115 11 Luke Murphey
var.textcriticalpath = "/Users/lmurphey/Documents/SP/Workspace/TextCritical.com"
116 11 Luke Murphey
117 11 Luke Murphey
# This is IP of the web-server:
118 11 Luke Murphey
var.proxyserver = "127.0.0.1"
119 11 Luke Murphey
120 11 Luke Murphey
# This is the port that Django server is running on:
121 11 Luke Murphey
var.proxyport = 8080
122 11 Luke Murphey
123 11 Luke Murphey
# This is the port to run the web-server on:
124 11 Luke Murphey
var.serverport = 8081
125 11 Luke Murphey
126 11 Luke Murphey
server.modules = ( "mod_proxy" )
127 11 Luke Murphey
128 11 Luke Murphey
$HTTP["url"] !~ "^/media/" {
129 11 Luke Murphey
	proxy.server = ( "" =>
130 11 Luke Murphey
	                       ( ( 
131 11 Luke Murphey
	                           "host" => var.proxyserver,
132 11 Luke Murphey
	                           "port" => var.proxyport
133 11 Luke Murphey
	                       ) )
134 11 Luke Murphey
	               )
135 11 Luke Murphey
}
136 11 Luke Murphey
137 11 Luke Murphey
server.document-root = var.textcriticalpath + "/src"
138 11 Luke Murphey
139 11 Luke Murphey
server.port = var.serverport
140 11 Luke Murphey
141 11 Luke Murphey
mimetype.assign = (
142 11 Luke Murphey
  ".html" => "text/html", 
143 11 Luke Murphey
  ".txt" => "text/plain",
144 11 Luke Murphey
  ".jpg" => "image/jpeg",
145 11 Luke Murphey
  ".png" => "image/png",
146 11 Luke Murphey
  ".css" => "text/css",
147 11 Luke Murphey
  ".js" => "application/javascript"  
148 11 Luke Murphey
)
149 11 Luke Murphey
</pre>
150 11 Luke Murphey
151 12 Luke Murphey
h2. Upgrading
152 12 Luke Murphey
153 12 Luke Murphey
If you are upgrading an existing install from one of the older versions that supported Django 1.5, then you will need to do several things.
154 12 Luke Murphey
155 12 Luke Murphey
First, install Django 1.8:
156 12 Luke Murphey
157 12 Luke Murphey
<pre>
158 12 Luke Murphey
pip install Django==1.8.6
159 12 Luke Murphey
</pre>
160 12 Luke Murphey
161 12 Luke Murphey
Next, migrate the database:
162 12 Luke Murphey
163 12 Luke Murphey
<pre>
164 12 Luke Murphey
python manage.py migrate
165 12 Luke Murphey
</pre>
166 15 Luke Murphey
167 15 Luke Murphey
When you upgrade a TextCritical instance, you may need to clear some caches including:
168 15 Luke Murphey
* The web cache
169 15 Luke Murphey
* The cache eBooks (in /media/files)
170 15 Luke Murphey
171 15 Luke Murphey
The cache of eBooks can be rebuilt by running the Django commands +make_epubs+ and +make_mobi+.
172 12 Luke Murphey
 
173 7 Luke Murphey
h2. Common Setup Issues
174 3 Luke Murphey
175 3 Luke Murphey
h3. Syncdb Fails Saying it couldn't create "django_content_type"
176 3 Luke Murphey
177 3 Luke Murphey
This has something to do with database routing. It can be solved by adding the following to the settings.py file and re-running syncdb:
178 3 Luke Murphey
179 3 Luke Murphey
<pre>
180 3 Luke Murphey
DATABASE_ROUTING = []
181 3 Luke Murphey
</pre>
182 3 Luke Murphey
183 5 Luke Murphey
Once syncdb succeeds, delete this line.
184 5 Luke Murphey
185 5 Luke Murphey
Reference: https://code.djangoproject.com/ticket/16039
186 5 Luke Murphey
187 3 Luke Murphey
h3. Syncdb Fails When Creating Superuser
188 3 Luke Murphey
189 3 Luke Murphey
<pre>
190 3 Luke Murphey
You just installed Django's auth system, which means you don't have any superusers defined.                                                 
191 3 Luke Murphey
Would you like to create one now? (yes/no): yes                                                                                             
192 3 Luke Murphey
Traceback (most recent call last):                                                                                                          
193 3 Luke Murphey
  File "manage.py", line 10, in <module>                                                                                                    
194 3 Luke Murphey
    execute_from_command_line(sys.argv)                                                                                                     
195 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line                  
196 3 Luke Murphey
    utility.execute()                                                                                                                       
197 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute                                    
198 3 Luke Murphey
    self.fetch_command(subcommand).run_from_argv(self.argv)                                                                                 
199 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv                                  
200 3 Luke Murphey
    self.execute(*args, **options.__dict__)                                                                                                 
201 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute                                        
202 3 Luke Murphey
    output = self.handle(*args, **options)                                                                                                  
203 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle                                         
204 3 Luke Murphey
    return self.handle_noargs(**options)                                                                                                    
205 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 110, in handle_noargs                       
206 3 Luke Murphey
    emit_post_sync_signal(created_models, verbosity, interactive, db)                                                                       
207 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/sql.py", line 189, in emit_post_sync_signal                           
208 3 Luke Murphey
    interactive=interactive, db=db)                                                                                                         
209 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 172, in send                                            
210 3 Luke Murphey
    response = receiver(signal=self, sender=sender, **named)                                                                                
211 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 73, in create_superuser                    
212 3 Luke Murphey
    call_command("createsuperuser", interactive=True, database=db)                                                                          
213 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 150, in call_command                               
214 3 Luke Murphey
    return klass.execute(*args, **defaults)                                                                                                 
215 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute                                        
216 3 Luke Murphey
    output = self.handle(*args, **options)                                                                                                  
217 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/management/commands/createsuperuser.py", line 70, in handle              
218 3 Luke Murphey
    default_username = get_default_username()                                                                                               
219 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 105, in get_default_username               
220 3 Luke Murphey
    default_username = get_system_username()                                                                                                
221 3 Luke Murphey
  File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/management/__init__.py", line 85, in get_system_username                 
222 3 Luke Murphey
    return getpass.getuser().decode(locale.getdefaultlocale()[1])                                                                           
223 3 Luke Murphey
TypeError: decode() argument 1 must be string, not None   
224 3 Luke Murphey
</pre>
225 3 Luke Murphey
226 3 Luke Murphey
You may need to set your locale by running the following before running syncdb:
227 3 Luke Murphey
228 3 Luke Murphey
<pre>
229 3 Luke Murphey
export LC_ALL="en_US.UTF-8"
230 3 Luke Murphey
</pre>
231 14 Luke Murphey
232 14 Luke Murphey
Delete the auth_users table and do another syncdb to try again.
233 14 Luke Murphey
234 14 Luke Murphey
Reference: https://code.djangoproject.com/ticket/16017
235 14 Luke Murphey
236 14 Luke Murphey
h3. "__init__() keywords must be strings" when running Perseus Importer
237 14 Luke Murphey
238 14 Luke Murphey
This is caused by a "known bug in Python 2.6":http://bugs.python.org/issue2646 in which unicode arguments are not handled correctly. Upgrading to Python 2.7 resolves the issue.